optimus prime: a new tool for interactive transformation and
TRANSCRIPT
The University of York Department of Computer Science
Submitted in part fulfilment for the degree of MEng
Optimus PrimeA new tool for interactive transformation and
supercompilation of functional programs
Jason S Reich
12th May 2009
Supervisor Professor Colin Runciman
Number of words = 19569 as counted by detex | wc -wNumber of pages = 70
This includes the body of the report and Appendix A and Appendix E but notother appendices the bibliography and intentionally blank pages
Abstract
This dissertation investigates the use of supercompilation to improve the performance offunctional programs We apply the transformation strategy to the f-lite subset of Haskell98 Supercompilation is found to be an effective optimisation for f-lite programs andareas for further improvements are suggested
An interactive transformation tool Optimus Prime is produced to aid the developmentof the supercompiler The tool facilitates the observation of program transformationstrategies in progress and we gain the ability to operate on programs directly by manuallyapplying transformations
A key technique used in the implementation of Optimus Prime is the SyntacticZipper structure We conclude that while the use of this data structure is nearly essentialfor interactive program transformation it becomes a performance limitation for fullyautomatic transformation strategies
Keywords functional programming program transformation interactive transformationsupercompilation zipper data structure Haskell Reduceron f-lite
4
Acknowledgements
Many thanks go to Professor Colin Runciman for his support guidance good humourand RunciScribbletrade through the course of this project
My gratitude to Matthew Naylor for providing the project setting and his assistance withrunning Reduceron 2 f-lite code
I am much obliged to Neil Mitchell his willingness to discuss his work on Supercompila-tion
A solemn nod to the many programs who willingly dedicated their lives to science onlyto be mangled into a non-executable form by early versions of Optimus Prime
Finally three cheers to the occupants of the CS001 Mastersrsquo Laboratory for their motiva-tional techniques and other interesting activities
6
Contents
1 Introduction 1511 Motivation 15
12 Aims of this Project 16
13 Structure of this Report 16
14 Statement of Ethics 17
2 Background and Review 1821 Functional Languages and Haskell 18
22 Program Transformation 21
221 The FoldUnfold Laws 22
222 Defunctionalisation 22
223 Deforestation and Fusion 24
224 Supercompilation 24
23 Program Transformation Tools 25
24 The Reduceron and F-Lite 28
25 Summary 30
3 Problem Analysis 3131 Specification of Transformations 31
32 The Core Language 31
321 Preliminary Definitions 33
33 Transformation Primitives for f-lite 34
331 Inline Unfold 34
332 Fold 35
333 Residuate 35
34 The FoldUnfold Laws 35
35 Supercompilation Strategy 38
351 Termination Criteria 38
352 Unfolding Selection 39
353 Simplification Transformations for f-lite 39
354 Residuation Folding and Generalisation 42
355 Inlining Unnecessary Residuals 43
36 Requirements A Transformation Framework 43
37 Summary 44
4 Design and Implementation 4641 Representation through Syntactic Zippers 46
42 Zipper Query and Modification 50
43 Transformation Specification 55
7
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
Abstract
This dissertation investigates the use of supercompilation to improve the performance offunctional programs We apply the transformation strategy to the f-lite subset of Haskell98 Supercompilation is found to be an effective optimisation for f-lite programs andareas for further improvements are suggested
An interactive transformation tool Optimus Prime is produced to aid the developmentof the supercompiler The tool facilitates the observation of program transformationstrategies in progress and we gain the ability to operate on programs directly by manuallyapplying transformations
A key technique used in the implementation of Optimus Prime is the SyntacticZipper structure We conclude that while the use of this data structure is nearly essentialfor interactive program transformation it becomes a performance limitation for fullyautomatic transformation strategies
Keywords functional programming program transformation interactive transformationsupercompilation zipper data structure Haskell Reduceron f-lite
4
Acknowledgements
Many thanks go to Professor Colin Runciman for his support guidance good humourand RunciScribbletrade through the course of this project
My gratitude to Matthew Naylor for providing the project setting and his assistance withrunning Reduceron 2 f-lite code
I am much obliged to Neil Mitchell his willingness to discuss his work on Supercompila-tion
A solemn nod to the many programs who willingly dedicated their lives to science onlyto be mangled into a non-executable form by early versions of Optimus Prime
Finally three cheers to the occupants of the CS001 Mastersrsquo Laboratory for their motiva-tional techniques and other interesting activities
6
Contents
1 Introduction 1511 Motivation 15
12 Aims of this Project 16
13 Structure of this Report 16
14 Statement of Ethics 17
2 Background and Review 1821 Functional Languages and Haskell 18
22 Program Transformation 21
221 The FoldUnfold Laws 22
222 Defunctionalisation 22
223 Deforestation and Fusion 24
224 Supercompilation 24
23 Program Transformation Tools 25
24 The Reduceron and F-Lite 28
25 Summary 30
3 Problem Analysis 3131 Specification of Transformations 31
32 The Core Language 31
321 Preliminary Definitions 33
33 Transformation Primitives for f-lite 34
331 Inline Unfold 34
332 Fold 35
333 Residuate 35
34 The FoldUnfold Laws 35
35 Supercompilation Strategy 38
351 Termination Criteria 38
352 Unfolding Selection 39
353 Simplification Transformations for f-lite 39
354 Residuation Folding and Generalisation 42
355 Inlining Unnecessary Residuals 43
36 Requirements A Transformation Framework 43
37 Summary 44
4 Design and Implementation 4641 Representation through Syntactic Zippers 46
42 Zipper Query and Modification 50
43 Transformation Specification 55
7
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
4
Acknowledgements
Many thanks go to Professor Colin Runciman for his support guidance good humourand RunciScribbletrade through the course of this project
My gratitude to Matthew Naylor for providing the project setting and his assistance withrunning Reduceron 2 f-lite code
I am much obliged to Neil Mitchell his willingness to discuss his work on Supercompila-tion
A solemn nod to the many programs who willingly dedicated their lives to science onlyto be mangled into a non-executable form by early versions of Optimus Prime
Finally three cheers to the occupants of the CS001 Mastersrsquo Laboratory for their motiva-tional techniques and other interesting activities
6
Contents
1 Introduction 1511 Motivation 15
12 Aims of this Project 16
13 Structure of this Report 16
14 Statement of Ethics 17
2 Background and Review 1821 Functional Languages and Haskell 18
22 Program Transformation 21
221 The FoldUnfold Laws 22
222 Defunctionalisation 22
223 Deforestation and Fusion 24
224 Supercompilation 24
23 Program Transformation Tools 25
24 The Reduceron and F-Lite 28
25 Summary 30
3 Problem Analysis 3131 Specification of Transformations 31
32 The Core Language 31
321 Preliminary Definitions 33
33 Transformation Primitives for f-lite 34
331 Inline Unfold 34
332 Fold 35
333 Residuate 35
34 The FoldUnfold Laws 35
35 Supercompilation Strategy 38
351 Termination Criteria 38
352 Unfolding Selection 39
353 Simplification Transformations for f-lite 39
354 Residuation Folding and Generalisation 42
355 Inlining Unnecessary Residuals 43
36 Requirements A Transformation Framework 43
37 Summary 44
4 Design and Implementation 4641 Representation through Syntactic Zippers 46
42 Zipper Query and Modification 50
43 Transformation Specification 55
7
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
Acknowledgements
Many thanks go to Professor Colin Runciman for his support guidance good humourand RunciScribbletrade through the course of this project
My gratitude to Matthew Naylor for providing the project setting and his assistance withrunning Reduceron 2 f-lite code
I am much obliged to Neil Mitchell his willingness to discuss his work on Supercompila-tion
A solemn nod to the many programs who willingly dedicated their lives to science onlyto be mangled into a non-executable form by early versions of Optimus Prime
Finally three cheers to the occupants of the CS001 Mastersrsquo Laboratory for their motiva-tional techniques and other interesting activities
6
Contents
1 Introduction 1511 Motivation 15
12 Aims of this Project 16
13 Structure of this Report 16
14 Statement of Ethics 17
2 Background and Review 1821 Functional Languages and Haskell 18
22 Program Transformation 21
221 The FoldUnfold Laws 22
222 Defunctionalisation 22
223 Deforestation and Fusion 24
224 Supercompilation 24
23 Program Transformation Tools 25
24 The Reduceron and F-Lite 28
25 Summary 30
3 Problem Analysis 3131 Specification of Transformations 31
32 The Core Language 31
321 Preliminary Definitions 33
33 Transformation Primitives for f-lite 34
331 Inline Unfold 34
332 Fold 35
333 Residuate 35
34 The FoldUnfold Laws 35
35 Supercompilation Strategy 38
351 Termination Criteria 38
352 Unfolding Selection 39
353 Simplification Transformations for f-lite 39
354 Residuation Folding and Generalisation 42
355 Inlining Unnecessary Residuals 43
36 Requirements A Transformation Framework 43
37 Summary 44
4 Design and Implementation 4641 Representation through Syntactic Zippers 46
42 Zipper Query and Modification 50
43 Transformation Specification 55
7
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
6
Contents
1 Introduction 1511 Motivation 15
12 Aims of this Project 16
13 Structure of this Report 16
14 Statement of Ethics 17
2 Background and Review 1821 Functional Languages and Haskell 18
22 Program Transformation 21
221 The FoldUnfold Laws 22
222 Defunctionalisation 22
223 Deforestation and Fusion 24
224 Supercompilation 24
23 Program Transformation Tools 25
24 The Reduceron and F-Lite 28
25 Summary 30
3 Problem Analysis 3131 Specification of Transformations 31
32 The Core Language 31
321 Preliminary Definitions 33
33 Transformation Primitives for f-lite 34
331 Inline Unfold 34
332 Fold 35
333 Residuate 35
34 The FoldUnfold Laws 35
35 Supercompilation Strategy 38
351 Termination Criteria 38
352 Unfolding Selection 39
353 Simplification Transformations for f-lite 39
354 Residuation Folding and Generalisation 42
355 Inlining Unnecessary Residuals 43
36 Requirements A Transformation Framework 43
37 Summary 44
4 Design and Implementation 4641 Representation through Syntactic Zippers 46
42 Zipper Query and Modification 50
43 Transformation Specification 55
7
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
Contents
1 Introduction 1511 Motivation 15
12 Aims of this Project 16
13 Structure of this Report 16
14 Statement of Ethics 17
2 Background and Review 1821 Functional Languages and Haskell 18
22 Program Transformation 21
221 The FoldUnfold Laws 22
222 Defunctionalisation 22
223 Deforestation and Fusion 24
224 Supercompilation 24
23 Program Transformation Tools 25
24 The Reduceron and F-Lite 28
25 Summary 30
3 Problem Analysis 3131 Specification of Transformations 31
32 The Core Language 31
321 Preliminary Definitions 33
33 Transformation Primitives for f-lite 34
331 Inline Unfold 34
332 Fold 35
333 Residuate 35
34 The FoldUnfold Laws 35
35 Supercompilation Strategy 38
351 Termination Criteria 38
352 Unfolding Selection 39
353 Simplification Transformations for f-lite 39
354 Residuation Folding and Generalisation 42
355 Inlining Unnecessary Residuals 43
36 Requirements A Transformation Framework 43
37 Summary 44
4 Design and Implementation 4641 Representation through Syntactic Zippers 46
42 Zipper Query and Modification 50
43 Transformation Specification 55
7
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
Contents
44 Transformation Specifics 57
441 Low-Level Definitions 57
442 Transformation Rules and Strategies 58
45 Output and Control 60
451 Pretty printing 60
452 Interactive Control 61
453 Autonomous Control 62
46 Summary 62
5 Results and Evaluation 6351 Performance of the Syntactic Zipper Library 63
52 Performance of the Supercompiler 63
521 Test Programs 64
522 The Supercompilation Process 65
523 The Supercompiled Programs 65
524 Discussion 65
53 Summary 67
6 Conclusions and Further Work 6861 Satisfaction of Objectives 68
62 Taking Zippers out of Context 69
63 Optimisation of the Homeomorphic Embedding Relation 69
64 Supercompilation of Primitive Functions 69
65 Closing Remarks 70
A FoldUnfold Transformation Transcripts 71A1 Fibonacci Transformation Transcript 71
A2 AppendElem Proof Transcript 72
B Tree Traversal Comparison 74
C Test Programs 77C1 Perms mdash Permiatation Counting 77
C2 Fibonacci mdash Calculation of Fibonacci Numbers 79
C3 Twizzler mdash The Twizzler Problem 80
C4 Word Count mdash Counting Words in a String 82
C5 N-Queens mdash N = 10 Instance 83
D Supercompilation Report for Word Count 85
E Supercompiled Word Count 90
Bibliography 92
8
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
List of Figures
31 An example of a tree transformation 32
32 Compact notation for the transformation of Figure 31 32
33 Syntax of our core language 33
34 The children of f-lite expressions 34
35 Definition of inline 35
36 Example of variable capture from a naive inlining 35
37 Definition of fold 36
38 Definition of residuate 36
39 Root expressions that will cause simple termination 38
310 The homeomorphic embedding relation homeo or E 39
311 The expression similarity relation sim 39
312 Simplification laws for f-lite 40
313 New and augmented simplification transformations for f-lite 41
314 Most specific generalisation rewrite rule 42
315 Requirements for Optimus Prime 45
41 Optimus Prime architecture 46
42 Example of a zipper structure 48
43 Example of scoping issues 48
44 The supercompilation algorithm 59
45 The interface of Optimus Prime 61
9
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
List of Figures
10
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
List of Tables
31 Transformation variable conventions 32
51 Comparison of tree query and modification techniques 64
52 Effects of supercompilation on code 66
53 Supercompilation simplification statistics 66
54 Performance of supercompiled programs 66
11
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
List of Tables
12
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-
List of Listings
21 Examples of higher order functions 19
22 Example of the effect of evaluation models 19
23 Infinite fibonacci sequence 20
24 Example of overloading leaves using a typeclass 20
25 Monad typeclass 21
26 The Maybe monad 21
27 Using foldunfold to compose two functions 23
28 Another example of a higher order function (Mitchell 2007) 23
29 Reynoldsrsquo style defunctionalisation (Mitchell 2007) 23
210 Mitchellrsquos style defunctionalisation (Mitchell 2007) 23
211 Non-treeless and treeless definitions of flatten (Wadler 1990) 24
212 Supercompiling to perform fusion 26
213 AST for the f-lite syntax 29
214 Examples of Naylorrsquos transformations 30
31 Examples of abstraction in f-lite 37
41 Syntactic zipper types 47
42 Primitive syntactic zipper operations 49
43 BindingMap data structure 50
44 Zipper structure for f-lite 51
45 Zipper query functions 51
46 Example of zipper query using list comprehensions 51
47 Zipper modification functions 52
48 Example of boilerplate code 53
49 Uniplate representation of Listing 48 53
410 Our zipper representation of Listing 48 54
411 Complex empty let removal for f-lite 55
412 Simpler empty let removal for f-lite 56
413 Monadic empty let removal for f-lite 56
414 VariableMap state 57
415 Preliminary definitions 58
A1 An example of optimising a function using foldunfold expressed in f-lite 71
A2 Example of a foldunfold proof held in f-lite syntax 72
B1 Code for tree traversal tests 74
C1 Code for the lsquopermsrsquo test program 77
C2 Code for the lsquofibonaccirsquo test program 79
13
List of Listings
C3 Code for the lsquotwizzlerrsquo test program 80
C4 Code for the lsquowcrsquo test program 82
C5 Code for the lsquonqueensrsquo test program 83
E1 Supercompiled code for the lsquoword countrsquo test program 90
14
1 Introduction
This project is concerned with interactive and automatic program transformation in afunctional language
Programs are descriptions of computations represented in a language with formalsyntactic and semantic models A transformation is an alteration of a programrsquos syntaxsuch that the transformed program is semantically equivalent to the original Transform-ations can introduce elementary changes such as the renaming functions and variablesor more elaborate changes such as optimisation for a platformrsquos characteristics
A variety of programming language styles are available The large majority of main-stream languages are described as imperative These emphasise a ldquostyle in which programsexecute commands sequentially use variables to organise memory and update variables withassignment statementsrdquo (Finkel 1995) Another less common paradigm is functional pro-gramming Computation is expressed as the evaluation of functions where the conceptsof an implicit state and side effects are eliminated
Proponents argue that beyond the novelty of functional languages ldquoprograms can bewritten quicker are more concise are higher level are more amenable to formal reasoning andanalysis and can be executed more easily on parallel architecturesrdquo (Hudak 1989) It is thecompliance with formal reasoning that makes this project feasible
A purely functional language provides referential transparency ldquoin which equals canbe replaced by equalsrdquo (Hudak 1989) This is because immutable functions will alwaysreturn the same result for a given input Transformations can be safely performedas any expression can be replaced with its definition The implications of referentialtransparency are discussed in Section 21
We apply this principle to produce a tool Optimus Prime that allows the user totransform a program interactively The transformations do not change the output of theprogram but do alter the execution characteristics
To demonstrate the power of such a tool we implement the supercompilation (Turchin1979) optimisation strategy and use it to improve the performance of programs automat-ically without human guidance
11 Motivation
Program transformations are often applied by a compiler to optimise readable code intoa more efficient form While reasoning about a transformation can prove its semanticequivalence one will still need to experiment with the application of the transformation(and transformation strategy) to find the conditions under which an improvement ismade
An interactive transformation tool allows a user to manually apply transformations Theresulting code will always be executable as semantic equivalence is maintained The usercan then observe the efficiency of the executing program at any given stage
15
1 Introduction
The Reduceron 2 (Naylor and Runciman 2008) is a FPGA graph reduction machinefor executing a subset of Haskell (Simon Peyton Jones et al 2003) f-lite The machine isa quite sophisticated in its approach but the current f-lite compiler is very simple andperforms only basic optimisation
In principle program transformation strategies can be applied to optimise code to runmore efficiently on this type of platform Supercompilation is one strategy that could beapplied Questions include (a) What extensions are required for it operate on the f-litelanguage and (b) Will lead to performance improvements on this architecture
12 Aims of this Project
This project aims to do the following
1 to develop an interactive transformation tool Optimus Prime that can operateon the f-lite subset of Haskell 98
2 to apply the foldunfold program transformations to the f-lite language to beused for intuition-guided optimisation theorem proving and as the basis of morecomplex program transformation strategies
3 to investigate the complications of applying supercompilation to the f-lite languageand where extensions may be required
4 to implement a supercompiler for the f-lite language using the Optimus Primeframework and
5 to measure the costs of supercompilation and investigate whether it produces adistinct improvement to the performance of f-lite programs
13 Structure of this Report
Chapter 2 mdash Background and Review A review of key concepts and existing literatureis presented Functional languages are introduced and particular features of interestare highlighted Program transformations specific examples and tools for performingtransformations are discussed Finally the reader is acquainted with the Reduceron 2
and the f-lite language
Chapter 3 mdash Problem Analysis Investigates how to formally define a program trans-formation A core language is defined and evaluated Primitive transformations thefoldunfold laws and the supercompilation strategy are applied to our core languageFinally we discuss the requirements for our transformation tool
Chapter 4 mdash Design and Implementation A data structure for representing a programinside an interactive transformation tool is presented Operations upon this data structureare defined and evaluated We discuss how to specify transformations and strategieswithin the tool Finally we investigate the formatting of programs to be returned by thetool and how the user may interact with the system
16
14 Statement of Ethics
Chapter 5 mdash Results and Evaluation We evaluate the performance of our Syntactic Zipperlibrary and its suitability for our needs The performance of the Supercompilation processis assessed in terms of speed of compilation and optimisation of programs
Chapter 6 mdash Conclusions and Further Work The projectrsquos objectives are revisited and oursuccess in satisfying them is assessed We conclude that there is indeed scope for usingsupercompilation to optimise programs for the Reduceron 2 platform Areas for furtherresearch to improve the speed of supercompilation and the performance of optimisedprograms are presented
14 Statement of Ethics
This project adheres to the ACM Code of Ethics (Anderson 1992) and departmentalguidelines The project is concerned with the production of a software tool where noexternal human participants were required for testing The tool is provided as-is andshould be considered of experimental quality
17
2 Background and Review
This chapter will introduce a number of the key concepts that are investigated andapplied through the course of this project Existing literature is examined to extract anybeneficial experience and knowledge
Section 21 demonstrates a selection of functional programming concepts and intro-duces the Haskell language The functional language constructs described are eithertargets for optimisation or structures used to implement the system
Transformations and their practical applications are discussed in Section 22 alongwith a few concrete examples of program transformations in functional languagesExisting tools for performing program in both autonomous and interactive fashions areinvestigated in Section 23
Finally in Section 24 the reader is introduced to Naylorrsquos (2008) Reduceron 2 and f-liteThe f-lite subset of Haskell is the language of programs programs to be transformed andthe Reduceron 2 is the architecture upon which the code will be executed
21 Functional Languages and Haskell
Functional languages Functional languages are characterised by de-emphasising oreradicating mutable data and lsquoside effectsrsquo by appealing to the mathematical definition ofa function
Proponents (Hudak 1989) argue that programs in functional languages are moreresponsive to formal reasoning and encourage more concise program code Constructs infunctional languages also map well onto parallel architectures
Several general purpose functional languages exist including Miranda (Turner 1990)ML (Milner et al 1990) LISP (Steele 1990) Erlang (Armstrong 2007) Clean (van Eekelenand Plasmeijer 2001) and Microsoft F (Syme) MATLAB XSLT and Microsoft Excelformulae are all examples of domain specific functional languages
Some of these languages borrow concepts from Church (1936) lambda calculus a formallogic describing functions applications and recursion In lambda calculus expressionsare reduced using three rules alpha beta and eta reduction α-conversion allows thenames of any bound variables to be changed β-reduction is function application bysubstituting a lambdarsquos variable with arguments η-equivalence is where two expressionsproduce the results
Haskell Haskell (Simon Peyton Jones et al 2003) is a ldquogeneral purpose purely functionalprogramming languagerdquo It combines a number of emerging and interesting concepts inthe field of functional languages such as higher order functions non-strict semanticsand pattern matching The subset f-lite (see Section 24) retains these features Monadsand typeclasses are Haskell constructs that are not included in f-lite but are used in ourimplementation (see Chapter 4)
18
21 Functional Languages and Haskell
map f [ ] = [ ]map f ( x xs ) = f x map f xs
f o l d r f z [ ] = zf o l d r f z ( x xs ) = f x ( f o l d r f z xs )
( ) f g x = f ( g x )
Listing 21 Examples of higher order functions
loop = loop
const x y = x
f = const 1 loopminusminus Under c a l lminusbyminusneed s e m a n t i c s f e v a l u a t e s t o 1 minusminus Under c a l lminusbyminusv a l u e s e m a n t i c s f d o e s not t e r m i n a t e
Listing 22 Example of the effect of evaluation models
Higher order functions A higher order function (or functional) is a function on a functionExamples include lsquomaprsquo lsquofoldrrsquo and function composition Example definitions of thesehigher order functions can be found in Listing 21
Lazy evaluation Non-strict evaluation is a reduction strategy where the outermost redu-cible expression is reduced first Haskell specifically uses graph reduction to ensurethat while expressions are only evaluated as required they are also only evaluated oncefor any shared use Other terms for this strategy include lazy delayed and call-by-needevaluation
In a call-by-value language the function f in Listing 22 would not terminate becauseit needs to evaluate loop before it can evaluate const A call-by-need language such asHaskell does not need the value of loop and terminates with the result 1
Call-by-need semantics ensure full referential transparency (discussed in Chapter 1)Inlining const in f leaves f = 1 This expression is equivalent to the original definitionof f under call-by-need semantics but not call-by-value The termination characteristicshave changed
Hudak et al (1999) highlights that a significant advantage of Haskellrsquos non-strict natureis that data constructors are also non-strict This allows the definition of infinite datastructures Despite the structure being infinite in size (taking an infinite amount of timeand space to calculate in its entirety) the needed parts can be computed in finite timeListing 23 illustrates an example of an infinite data structure holding the Fibonaccisequence
Typeclasses Typeclasses are Haskellrsquos interpretation of function overloading A typeclassdefines a ldquoset of functions that can have different implementations depending on the type of datathey are givenrdquo (OrsquoSullivan et al 2008) Listing 24 shows a typeclass Tree that defines
19
2 Background and Review
f i b = 1 1 sumTwo f i b
sumTwo ( x y zs ) = x + y sumTwo ( y zs )
Listing 23 Infinite fibonacci sequence
data BinaryTree = Branch2 BinaryTree BinaryTree| Leaf2 I n t
data TernaryTree = Branch3 TernaryTree TernaryTree TernaryTree| Leaf3 I n t
c l a s s Tree a wherel eaves a rarr [ I n t ]
instance Tree BinaryTree wherel eaves ( Leaf2 x ) = [ x ]leaves ( Branch2 x y ) = leaves x ++ leaves y
instance Tree TernaryTree wherel eaves ( Leaf3 x ) = [ x ]leaves ( Branch3 x y z ) = leaves x ++ leaves y ++ leaves z
breadth Tree a rArr a rarr I n tbreadth t = length ( leaves t )
Listing 24 Example of overloading leaves using a typeclass
a function leaves that returns the list of integers for a Tree instance Instances ofTree are provided for BinaryTree and TernaryTree Another function breadthis implemented using existing definitions of leaves Notice from its type signaturebreadth accepts any instance of Tree as an input
Monads Monads encapsulate computation in a data type ldquoMonads allow the programmerto build up computations using sequential building blocks which can themselves be sequences ofcomputationsrdquo (Newburn) Formally a monad consists of a type constructor a function tochain one monad onto a function producing the next (a binding operation) and a functionto inject a normal value into a monad chain (a unit function)
In Haskell these notions are implemented through the Monad typeclass (Listing 25)An example is given in Listing 26 of the Maybe monad which embodies partial functionsWhen the output of one function in a chain returns Nothing this result is propagatedthrough to the output Otherwise the value returned by a previous function will beused as the input to the next The Maybe monad will be used in Section 43 to handletransformation applicability The State monad will also be employed to provide asupply of fresh variable names
20
22 Program Transformation
c l a s s Monad m whereminusminus Binding o p e r a t i o n( gt gt=) m a rarr ( a rarrm b ) rarrm bminusminus Unit f u n c t i o nre turn a rarrm a
Listing 25 Monad typeclass
data Maybe a = J u s t a| Nothing
instance Monad Maybe whereNothing gtgt= f = NothingJ u s t x gtgt= f = f xreturn x = J u s t x
Listing 26 The Maybe monad
22 Program Transformation
A program transformation is ldquothe act of changing one program into anotherrdquo (Visser et al2004) More specifically there is a specific formal mapping of the program constructs inone form into another
Transformation has long been used to increase the efficiency of programs during thecompilation phase The popular GNU Compiler Collection (Free Software Foundation2009) includes loop optimisation common subexpression elimination and dead codeelimination transformations
The Glasgow Haskell Compiler (Haskellorg 2008) includes inlining specialisation anddeforestation transformation strategies GHC also uses normalisation transformations tolsquodesugarrsquo high-level language constructs into a lower level lsquocorersquo Haskell syntax
In addition to optimisation transformations have also been used for turning a program-ming written in one language into another (migration) moving from abstract specificationto a concrete program (synthesis and refinement) changing the readability of the programcode (refactoring and obfuscation) and reasoning about programsrsquo execution (Visser et al2004)
Burstall and Darlington (1977) foldunfold introduced in Subsection 221 is a set ofprimitive program transformations These can be applied to optimise a program orperform reasoning but there is no specific goal or strategy A number of heuristics aresuggested for optimising programs but they are not deterministic
In contrast defunctionalisation (Subsection 222) deforestation (Subsection 223) fusion(Subsection 223) and supercompilation (Subsection 224) are all transformation strategiesthat use the primitive laws of foldunfold to move code into specific forms
Defunctionalisation eliminates higher order functions Deforestation and Fusionremove intermediary data structures Supercompilation attempts to execute the programas far as possible at compile time without knowing the particular inputs Through thisprocess it achieves some of the effects of defunctionalisation deforestation and fusion
21
2 Background and Review
221 The FoldUnfold Laws
Burstall and Darlington (1977) introduced a series of inference rules under which recurs-ive programs can be reasoned about and optimised These laws have become collectivelyknown as the lsquofoldunfold rulesrsquo They are
1 Definition mdash The introduction of a new equation
2 Instantiation mdash The introduction of instances of an existing equation
3 Unfolding mdash Replacing an instance of a lsquoleft hand sidersquo with its corresponding lsquorighthand sidersquo definition
4 Folding mdash Replacing an instance of a lsquoright hand sidersquo with its corresponding lsquolefthand sidersquo definition
5 Abstraction mdash The extraction of expressions to introduce new local definitions
6 Laws mdash Appealing to any known theorems about the functions such as associativitycommutativity and distributively
These laws were applied by (Firth 1990) to a non-strict polymorphically typed patternmatching functional language Glide Firth highlighted and solved several issues thatarose from using foldunfold in this context
For example the non-strict semantics of Glide raises issues with the foldunfold lawsThe instantiation law replaces lazy variables with strict patterns Therefore regardless ofthe strictness of a function definition an instantiated form (using Burstall and Darlington(1977) will be) Additional work is required to preserve the strictness of a function Firth(1990) also introduces techniques for preserving the types and typability of functions andpreserving the definedness of functions
The application of foldunfold laws is illustrated in Listing 27 In this example twoHaskell functions are composed into one Burstall and Darlington (1977) do suggest anumber of heuristics to select which rule to apply when optimising programs Howeverno specific optimised strategy is presented for foldunfold
222 Defunctionalisation
Higher order functions were discussed in Section 21 Mitchell (2009) states that ldquohavingfunctions as first-class values leads to more concise code but it often complicate analysis meth-odsrdquo A technique that transforms programs written in a higher order style into firstorder would have obvious advantages
Reynolds (1972) eliminated higher order functions to simplify the interpretation offunctional code His technique involved encoding functions as data which are thenmapped onto the correct function definition at application Listing 29 shows howReynoldsrsquo technique would transform the higher order function heads in Listing 28
However Mitchell (2009) highlights that while Reynoldsrsquo technique is a reasonablysimple and complete transformation it actually makes the code more complex byadding indirection Mitchell introduces another technique for defunctionalisation usingsimplification inlining and specialisation program transformations The effect of thisstyle of defunctionalisation is shown in Listing 210
22
22 Program Transformation
minusminus O r i g i n a l d e f i n i t i o n ssum [ ] = 0 minusminus ( 1 )sum ( x xs ) = x + sum xs minusminus ( 2 )squares [ ] = [ ] minusminus ( 3 )squares ( x xs ) = x x squares xs minusminus ( 4 )sumSquares xs = sum ( squares xs ) minusminus ( 5 )
minusminus I n s t a n t i a t e ( 5 ) wi th xs = [ ] and xs = ( x xs )sumSquares [ ] = sum ( squares [ ] ) minusminus ( 6 )sumSquares ( x xs ) = sum ( squares ( x xs ) ) minusminus ( 7 )
minusminus Unfold s q u a r e s in ( 6 ) and ( 7 )sumSquares [ ] = sum [ ] minusminus ( 8 )sumSquares ( x xs ) = sum ( x x squares xs ) minusminus ( 9 )
minusminus Unfold sum in ( 9 )sumSquares ( x xs ) = x x + sum ( squares xs ) minusminus ( 1 0 )
minusminus Unfold sum in ( 8 ) Fo ld b a c k i n t o sumSquares in ( 1 0 ) us ing ( 5 ) sumSquares [ ] = 0 minusminus ( 1 1 )sumSquares ( x xs ) = x x + sumSquares xs minusminus ( 1 2 )
Listing 27 Using foldunfold to compose two functions
map f [ ] = [ ]map f ( x xs ) = f x map f xs
heads xs = map head xs
Listing 28 Another example of a higher order function (Mitchell 2007)
data Func = Headapply Head x = head xmap f [ ] = [ ]map f ( x xs ) = apply f x map f xsheads xs = map Head xs
Listing 29 Reynoldsrsquo style defunctionalisation (Mitchell 2007)
map_head [ ] = [ ]map_head ( x xs ) = head x map_head xsheads xs = map_head xs
Listing 210 Mitchellrsquos style defunctionalisation (Mitchell 2007)
23
2 Background and Review
minusminus Nonminus t r e e l e s s d e f i n i t i o nf l a t t e n [ [ a ] ] rarr [ a ]f l a t t e n [ ] = [ ]f l a t t e n ( xs xss ) = xs ++ f l a t t e n xss
minusminus T r e e l e s s d e f i n i t i o nf l a t t e n _ d [ [ a ] ] rarr [ a ]f l a t t e n _ d [ ] = [ ]f l a t t e n _ d ( xs xss ) = f l a t t e n _ d rsquo xs xss
f l a t t e n _ d rsquo [ a ] rarr [ [ a ] ] rarr [ a ]f l a t t e n _ d rsquo [ ] xss = f l a t t e n _ d xssf l a t t e n _ d rsquo ( x xs ) xss = x f l a t t e n _ d rsquo xs xss
Listing 211 Non-treeless and treeless definitions of flatten (Wadler 1990)
223 Deforestation and Fusion
Deforestation is a technique used to remove intermediate data structures those that areconstructed by one function only to be traversed by the next The example used forcomposition in Listing 27 is actually a form of deforestation The original definition ofsumSquares would construct a list of the squares only to then take it apart again tosum the elements The derived definition does not construct this intermediate list
Originally proposed by Wadler (1990) the deforestation transformation strategy spe-cifically removes intermediate trees by converting composed functions into a lsquotreelessrsquoform The difference between the non-treeless and treeless forms is demonstrated inListing 211
Wadler distinguishes between pure deforestation under which no intermediate valuesare permitted and blazed deforestation which allows some atomic intermediate values toremain Under the blazed deforestation scheme non-tree typed expressions are markedand prevented from composition
Deforestation was refined into a technique called fusion by Chin (1992) He differentiatesbetween the producer function that returns the data structure as output and the consumerfunction which accepts the data structure as an input In the expression c(p(x)) p isthe producer and c is the consumer
Chin (1992) states that a composition can be safely fused ldquoif the transformation sequencewhich follows does not going into a loop and there is no loss of efficiencyrdquo A composition can befused effectively if the intermediate data structure is removed resulting in a performanceimprovement Safe fusion is essential while effective fusion is merely desirable
Similar to blazing sub-expressions are tagged as safe or unsafe producers and con-sumers Safe fusion can be ensured by only fusing if both the producer and the consumerare safe Otherwise the algorithm will fail
224 Supercompilation
Supervised compilation or supercompilation (Turchin 1979) attempts to execute theprogram at compile time to produce a more efficient form As the inputs are unknown at
24
23 Program Transformation Tools
this time it produces lsquoresidualrsquo function definitions at each stage where it cannot proceedany further without knowing the input
Supercompilation achieves many of the effects of Wadler (1990) deforestation andChin (1992) fusion Listing 212 illustrates the deforestationfusing abilities of supercom-pilation Further examples will be presented as we investigate the application of thisalgorithm to our core language in Section 35
The algorithm begins by driving through a function definition inlining applications andsimplifying the results When it appears that the process is revisiting previously derivedresults (a condition known as lsquothe whistlersquo) the children of the outermost expression aregeneralised where possible The generalised forms are replaced by applications of existingequivalent functions or new functions called residuals The residuals are themselvesdriven through The process continues until no new residuals are generated
Soslashrensen et al (2008) restricts the supercompiler to only propagate positive informationand ignore negative information Take the example of an equality test in a conditionalonly the true branch has variables substituted as it is easier to pass the positive informa-tion the known value of the variable than it is to communicate negative information thevalue that the variable is not
The positive supercompiler is investigated again by Jonsson and Nordlander (2009) inthe setting of a call-by-value language They deal with the strictness issues discussed inSection 21 by performing strictness analysis on functions before inlining Functions thathave possibly non-strict arguments are prevented from being inlined
Mitchell (2008 chapter 4) investigates the application of supercompilation to a coresubset of Haskell He introduces a new generalisation technique that appears to producebetter performing residual programs than if the generalisation of Soslashrensen et al (2008) isapplied
Mitchell shows optimisations across the lsquoimaginaryrsquo section of the NoFib benchmarksuite (Partain 1992) He presents the startling instance of a supercompiled Haskellprogram performing faster than its C equivalent Mitchellrsquos work will be discussedfurther in Chapter 3 Problem Analysis
23 Program Transformation Tools
Visser et al (2004) state that ldquoone of the aims of a general framework for program transformationis to define transformations that are reusable across as wide a range of languages as possiblerdquo
Transformation metalanguages One technique for specifying transformations is the useof a formal metalanguage to describe transformations and the languages they operateupon Using these preliminary definitions a transformation tool can parse a program tobuild a abstract syntax tree perform transformation operations on this tree and output thenewly derived program These metalanguages are generally wide-spectrum to supportthe description of a multitude of programming language paradigms
The TXL project (Cordy et al 1988) is an example of such a tool An arbitrary contextfree-grammar is provided for the language to be transformed and transformation rulesare supplied in a functional form A variety of language descriptions in the TXL grammarformat are are freely available for common (generally imperative) languages allowing thespeedy implementation of new transformations
25
2 Background and Review
map f xs = case xs of[ ] rarr [ ]( y ys ) rarr f y map f ys
main xs = map ( ( 1 +) (map (2 ) xs )=== R e s i d u a t e
main xs = h0 xs
h0 xs = map (1 +) (map (1 ) xs )=== I n l i n e lsquomap lsquo
h0 xs = case (map (2 ) xs ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== I n l i n e lsquomap lsquo h0 xs = case ( case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (2 ) y2 map (2 ) ys2 ) of
[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c a s e h0 xs = case xs of
[ ] rarr case [ ] of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
( y2 ys2 ) rarr case (2 ) y2 map (2 ) ys2 ) of[ ] rarr [ ]( y1 ys1 ) rarr (1 +) y1 map (1 +) ys1
=== Case o f c o n s t r u c t o r t w i c e h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr l e t y1 = (2 ) y2
ys1 = map (2 ) ys2 )in (1 +) y1 map (1 +) ys1
=== I n l i n e nonminusl i n e a r l e t s h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) map (1 +) map (2 ) ys2 )
=== Homeomorphic embedding t e r m i n a t i o n f o l d h0 xs = case xs of
[ ] rarr [ ]( y2 ys2 ) rarr (1 + (2 y2 ) ) h0 ys2
Listing 212 Supercompiling to perform fusion
26
23 Program Transformation Tools
Cordy (2006) highlights that transformation systems need to provide a method forcontrolling the direction of tree traversal Some transformations such as inlining afunction body at an application can lead to non-termination if executed in a top-downmanner Others require a bottom up movement
FermatT (Ward and Zedan 2005) StrategoXT (Visser 2004) DMS (Baxter et al 2004)and ASF+SDT (van den Brand et al 2001) are examples of other transformation systemsthat uses domain specific languages Each has their own particular implementationspecifics but overall they follow the same concept of describing the language and thentransformations upon it
An issue with such systems is due to their generic nature error messages can bequite vague ldquoOne problem with the general approach of language implementation bypreprocessor is the difficulty in providing error diagnostics that are related to the originalsource when the base programming language processor detects errors in the transformedresultrdquo (Cordy et al 1988) This makes debugging parsing and transformation errorsunnecessarily complicated
Another approach is the Template Haskell (Sheard and Jones 2002) meta-programmingsystem Haskell acts as both the language being operated upon and the languageperforming the operations One gains the benefits of using a transformation system thatis embedded in an existing robust compiler with familiar behaviour and error responsesBut one loses the ability to operate on other languages
Pattern matching and boilerplate removal Another approach is to implement a trans-formation system in a host general purpose programming language Parsing librariesfor reading in programs in arbitrary lsquotargetrsquo languages already exist for most popularprogramming languages Similarly pretty printing libraries exist for outputting theresults
For the actual transformation it is necessary to have a system that can traverse theabstract syntax tree and match patterns within the tree for replacement This is where ahost language that supports tree data type pattern matching has clear advantages As wehave already discussed such pattern matching features are fully supported Haskell 98
Custom functions could be written for performing tree traversal This lsquoboilerplatersquo codewould need to be rewritten for every variation of abstract syntax tree representation
Several techniques are available to alleviate the burden of having to write the oftentedious and repetitive definitions lsquoScrap Your Boilerplatersquo (Laumlmmel and Peyton Jones2003) introduces generic traversals where common traversal functions can be applied toany tree data type This technique requires extensions to Haskell 98 such as rank-2 typesand utilises derivable type classes to relieve the user from having to write any additionalcode to use the generic operations
Uniplate (Mitchell and Runciman 2007 Mitchell 2008 chap 3) is another take onboilerplate removal It abstracts all traversal operations to functions that utilise a singleoverloaded function uniplate that is implemented as part of the Uniplate typeclassOnce an instance of the Uniplate class any homogenous data type can be traversedusing standard functions Uniplate will be discussed further in Section 42 and comparedwith the solution for tree traversal in Optimus Prime
27
2 Background and Review
Interactive Transformation Major projects such as CIP (Bauer 1979) and KIDS (Smith1991) promoted the use of interactive program transformation systems to derive pro-grams from formal specifications The advantage of such a process would be thatconformity to a specification can be ensured due to formal semantic equivalence
Similar to the transformation metalanguages mentioned above the Computer-aidedIntuition-guided Programming project (Bauer 1979) uses a wide-spectrum languageCIP-L to define target languages and a related language CIP-S to define transformationsupon it While the transformation specifications include lsquoapplicability conditionsrsquo thesystem does not automatically apply the transformations Instead a human guidesthe process by entering transformation commands through a command-line interfaceDespite the human direction CIP ensures that transformations are performed correctly
CIP was designed to be as generic and flexible as possible to cover all transformationneeds in any given language Feather (1982) introduced ZAP an interactive system forimproving program performance by transformation ZAP was designed for optimisingexisting programs Hierarchies of transformations are developed from complex trans-formation strategies down to transformation primitives A control language is used toapply these transformations and can be entered interactively at the command-line orbatch scripted in advance and applied all in one go
STARSHIP Firth (1990) is an interactive transformation for operating on the Glide non-strict functional language It is built upon the Burstall and Darlington (1977) foldunfoldlaws with modifications for the new lazy functional setting (see Subsection 221) LikeZAP a control language is entered at a command like to control transformations Howeveras STARSHIP is considered a lsquoproof assistantrsquo advanced functionality is supportedfor ldquoexploratory transformationrdquo One such feature is backtracking through previouslyintroduced lsquolawsrsquo
Programmer Assistant for Transforming Haskell (PATH) (Tullsen 2002) is anotherldquouser-directed program transformation systemrdquo for use with a non-strict functional languageAs the name suggests it is specifically targeted at the Haskell language Like STARSHIPit builds upon the principles of FoldUnfold transformation while dealing with some ofthe issues caused by Haskellrsquos advanced features
The system is embedded in the Emacs text editor providing a dialog-based interfaceThis is a break from the command-line interfaces we have seen previously Tullsen claimsthat this is to improve its appeal to practitioners ldquoThe goal is for PATH to be usable by anynovice functional programmerrdquo (Tullsen 2002) This should be a goal for any interactiveprogram transformation tool An intuitive interface is essential
24 The Reduceron and F-Lite
The Reduceron 2 (Naylor and Runciman 2008) is a graph reduction machine for executinga functional language f-lite It is implemented on a field programmable gate array (FPGA)and features parallel wide memories for program code heap and stack
Most mainstream computer processors are engineered to execute imperative programsFunctional programs are transformed by complex compilers from their functional formsto execute efficiently on these platforms The Reduceron is being developed to researchthe advantages of utilising a special-purpose processor to execute functional code ratherthan the other way around
28
24 The Reduceron and F-Lite
minusminus A b s t r a c t sy n t ax t r e e r o o t L i s t o f f u n c t i o n d e c l a r a t i o n s type Prog = [ Decl ]
minusminus A f u n c t i o n d e c l a r a t i o n i n c l u d i n g a name arguments and e x p r e s s i o n data Decl = Func funcName Id
funcArgs [ Pat ] funcRhs Exp
minusminus A v a r i a b l e f u n c t i o n or c o n s t r u c t o r i d e n t i f i e rtype Id = S t r i n g
minusminus fminus l i t e e x p r e s s i o n sdata Exp = App Exp [ Exp ]
| Case Exp [ Alt ]| Let [ Bind ] Exp| Var Id| Con Id| Fun Id| I n t I n t| Bottom
minusminus P a t t e r n s f o r c a s e a l t e r n a t i v e s and arguments type Pat = Exp
minusminus Case a l t e r n a t i v e type Alt = ( Pat Exp )
minusminus L e t b i n d i n gtype Bind = ( Id Exp )
Listing 213 AST for the f-lite syntax
The f-lite language is a subset of Haskell 98 (Simon Peyton Jones et al 2003) consistingof function definitions case expressions limited let expressions function applicationsand constructor applications expressed in the explicit lsquobracesrsquo layout-insensitive formatNotably it excludes comments type definitions type checking and explicit lambdaexpressions
The language is complete Any Haskell constructs that are not included in the corelanguage can be desugared into an f-lite form The reduced language merely simplifiesthe development of the Reduceron Many examples of f-lite source listings are providedthroughout this project and the language is discussed in detail with reference to ourneeds in Section 32
Naylor and Runciman have released the code for an early iteration of the Reduceroncompiler written in Haskell It includes an f-lite parser a few transformation-basedoptimisations post-transformation pretty-printed output and a bytecode generator TheReduceron compiler represents a program abstract syntax tree using the data structuresoutlined in Listing 213
The parser is implemented using applicative parser combinators similar to thosedescribed in OrsquoSullivan et al (2008) This style benefits from a compact but readable
29
2 Background and Review
append Nil ys = l e t zs = ys in zs append ( Cons x xs ) ys = cons x ( append xs ys )
cons x xs = Cons x xs
minusminus a f t e r e q u a t i o n p a t t e r n d e s u g a r and l e t i n l i n i n g becomes minusminus
append v1 ys = case v1 of Nil rarr ys Cons x xs rarr cons x ( append xs ys )
cons x xs = Cons x xs
Listing 214 Examples of Naylorrsquos transformations
representation The compiler outputs the abstract syntax tree after the transformationphase for analysis and debugging Our reuse of this parser is discussed in Section 41
The output is produced through implementations of the lsquoShowrsquo typeclass for the ab-stract syntax tree data structures However this approach was considered to cumbersomefor our projects needs and an alternative will be discussed in Subsection 451
A number of transformations are already implemented in the Reduceron compilerincluding de-sugaring equation patterns to cases de-sugaring multiple constructor pat-terns to nested cases and inlining some let bindings The effects of these transformationsare illustrated by Listing 214
25 Summary
In this chapter we have introduced the language that we shall transform f-lite and thelanguage in which we shall implement our tool Haskell We have looked at relevantfeatures of these languages in preparation for the Problem Analysis and Design andImplementation chapters which follow
Program transformations and specific applications of program transformations havebeen discussed Program transformation strategies range from the low-level Burstall andDarlington (1977) foldunfold laws and to the powerful Turchin (1979) supercompilationstrategy We demonstrate how supercompilation can produce many of the desirableeffects of other optimisation strategies
We have classified existing program transformation tools into three groups (i) trans-formation metalanguages which use domain specific programming languages to describetarget languages and transformations (ii) pattern matching which exploits native featuresin general purpose programming languages to perform transformations and (iii) interact-ive transformation which use human intuition to guide the transformation process
Every transformation tool investigated has highlighted the need to be able to controlthe direction of traversal for transformation application Another important point to takeaway from the literature on interactive transformation is that a tool should ldquobe usable byany novice functional programmerrdquo (Tullsen 2002)
30
3 Problem Analysis
This chapter considers the issues and prerequisites for developing an interactive trans-formation system and implementing a selection of transformations within such a system
Section 31 presents a formalisation for specifying transformations to be included inour system The core language that our system will operate on is defined in Section 32and the transformation primitives for that language are discussed in Section 33
We shall investigate how the concepts of Burstall and Darlington (1977) foldunfoldtransformation (Section 34) and Turchin (1979) supercompilation (Section 35) can beapplied to our language
Finally in Section 36 the requirements for an interactive transformation system arederived from existing literature and our previous analysis
31 Specification of Transformations
A program transformation can be viewed as a manipulation of the abstract syntax treeFigure 31 illustrates an example of tree transformation Circular objects represent knownnodes Single bordered triangular objects are a placeholders for subtrees of any formDouble bordered triangular objects are placeholders for sequences of subtrees of anylength
A more compact notation for the same transformation is shown in Figure 32 Nodesare written in bold while transformation variables (placeholders) as written as x or xs Table 31 summarises the naming convention for transformation variables
32 The Core Language
As this project is orientated towards the development of compiler transformations forthe Reduceron our core language will be a normalised form of Naylorrsquos f-lite languageA normalised desugared format is used to aid the description of transformations Thesyntax is illustrated in Figure 33
As one of Naylorrsquos (Naylor and Runciman 2008) own compiler transformationsconverts pattern matching for function applications into case expressions patterns canbe omitted from function definitions Instead a simpler flat variable argument list isused This allows simpler transformation operations as will be demonstrated later
Similarly Naylorrsquos compiler desugars complex case patterns containing more thanone constructor or wild-card matches into nested case expressions each dealing witha single constructor component Therefore our core language can omit these lsquocomplexrsquopatterns only permitting the simple type
Comments are permitted at the declaration level to allow syntax trees to be annotatedwith information about the transformations that have taken place
31
3 Problem Analysis
A
B C
E
D
x ys
F
Dx
ys
Figure 31 An example of a tree transformation
A ( B E )( C x )( D ys )=rArr E x ( D ys )
Figure 32 Compact notation for the transformation of Figure 31
Table 31 Transformation variable conventions
Variables Description f-lite Examplesx y z Expression lsquo let x = inc 2 in D x x rsquo
v Variable lsquo v rsquo lsquo var1 rsquo lsquo apples rsquovL wL Local Variable vL sube v
vF Fresh Variable vF sube vc Constructor lsquo Nil rsquo lsquo Cons rsquo lsquo Record2 rsquof Function lsquo f rsquo lsquo (+) rsquo lsquo incBy1 rsquo
f P Primitive lsquo (+) rsquo lsquo (==) rsquo lsquo emit rsquof F Fresh Function f F sube f
m n Integer lsquo 0 rsquo lsquo 5 rsquo lsquo 111 rsquop Pattern lsquo v rsquo lsquo Nil rsquo lsquo( Cons x xs )rsquob Binding lsquo x = 10 rsquoa Alternative lsquo Nil rarr 10 rsquod Declaration lsquo inc x = (+) x 1 rsquo lsquo- blah -rsquo_s Sequence vs is a sequence of variables
32
32 The Core Language
prog = d1 dn
decl = f vs = x (function declaration)| minus ltstringgt minus (comment)
expr = ( x ) | v | c | f|m (integer)| x xs (application)| let v1 = x1 vn = xn in y (let)| case x of p1 rarr y1 pn rarr yn (case)
pat = c vs
Figure 33 Syntax of our core language
The core language includes seven primitive functions Addition (+) and subtraction(minus) operate on integers and return integers Equality (==) inequality (=) and lessthan or equal (lt=) operate on integers and return one of the zero-arity constructorsTrue and False The functions emitInt and emit output their argument and displaythe value of that argument as a benign side-effect Any application of a primitive functionmust be saturated
This core language differs from that used by Mitchell (2008) (i) it does not includeanonymous lambda abstractions (ii) it has primitive functions (iii) and it allows multiplebindings within a let expressions These differences must be taken into considerationwhen define simplification rules for the f-lite language as we will discuss in Section 35
Where more than one global declaration has been made for the same function nameonly the first definition is considered lsquoactiversquo This convention allows historical informa-tion about derivations to be stored without having to augment the abstract syntax treestructure
The semantics of the f-lite language allow for variable names to be reused in differentscopes Therefore any operation using variable names to identify specific variables mustensure that the variable name at this point refers to the intended variable One possiblefault that may occur is known as variable caputure An example is shown in Figure 36and the concept explained in Subsection 331
321 Preliminary Definitions
Bound variables The bound variables of an expression x are all those that are bound toa let expression binding or a case expression alternative pattern
33
3 Problem Analysis
children( x x1 x2 xn )= x x1 x2 xn
children( let v1 = x1 vn = xn in y )= x1 x2 xn y
children( case x of p1 rarr y1 pn rarr yn )= x y1 y2 yn
children(_) =
Figure 34 The children of f-lite expressions
Free variables Conversely the free variables of an expression x are all those within xthat are not bound to a let expression binding or a case expression alternative patternThe function f ree(x) returns the free variables of expression x
Substitution A substitution of a variable for an expression is denoted by x [ v y ] Allfree instances of v in x are replaced with expression y
Operations on sequences Collections of expressions can have a number of operationsapplied to them The length of a sequence length( xs ) is the number of itemscontained append( xs ys ) appends two sequences of expressions The zip opera-tion zip(v1 v2 vm x1 x2 xn) tuples together each element such that it re-turns v1 x1 v2 x2 vm xm when m le n The rest operation returns the elementsfrom the xs sequence not used in the zip operation That is to say rest( vs xs ) =xm+1 xm+2 xn
Relationships between expressions The children(x) are all the expressions that it con-tains as defined in Figure 34 Conversely the parent(x) is such that forallx children(p) middotparent(x) = p The descendants(x) = children(x) cup descendants(children(x))
33 Transformation Primitives for f-lite
331 Inline Unfold
An inline or unfold replaces function application with the corresponding instance ofthe function body A partial application occurs where fewer arguments are supplied thanthe arity of the function Inlining partial applications is not possible due to the lackof lambda abstractions in our core language Full application occurs when the numberof arguments and the arity are equal and over application occurs where the number ofarguments exceeds the arity
If inlining simply replaces variables by argument expression sharing may be lostwhere a variable is used more than once To preserve sharing a let expression is used to
34
34 The FoldUnfold Laws
f ys =rArr ( let zip( vs ys ) in y ) rest( vs xs )
Figure 35 Definition of inline where f vs = y and length vs le xs
f x = let y = 5 in (+) x y g y = f y
6equiv g y = let y = 5 in (+) y y
Figure 36 Example of variable capture from a naive inlining
bind local variables to the arguments Figure 35 gives the transformation specificationfor inline
Variable capture may occur when the inlined expression defines variables using namesthat have already been used Figure 36 gives an example of variable capture occurring Asimple technique to alleviate the issue is to give all bound variables in y lsquofreshrsquo (unused)names before the arguments are substituted This is a form of α-conversion
332 Fold
A transformational fold (not to be confused with the higher order function) can be seenas the inverse of inling If an expression is an instances (see Section 21) of a functionbody it can be replaced with a function application
333 Residuate
Residuation moves expressions into separate function definitions and can be seen as aform of abstraction (see Subsection 221) The expression is replaced with an applicationof the newly created function The free variables of the function must be passed into tothe new function as arguments Figure 38 depicts the transformation
The residuation process used by the Supercompilation strategy only produces re-siduals under carefully controlled circumstances The conditions will be described inSubsection 354
34 The FoldUnfold Laws
The foldunfold laws of Burstall and Darlington (1977) are useful for both the optimisa-tion of code and the reasoning about programs Originally conceived to operate on a firstorder recursion equation language they can be extended to operate on a non-strict high
35
3 Problem Analysis
x =rArr f ys
Figure 37 Definition of fold where f vs = y and y [ zip vs ys ] equivα x
x =rArr f F f ree(x)
and a new function definition is added for f F f ree(x) = x
Figure 38 Definition of residuate where f F is a fresh function name
order functional language This section will also consider how to express foldunfoldconcepts in our core language
Listing A1 in Appendix A illustrates the application of number of the laws to producean optimised f-lite definition of a Fibonacci series generator This is similar to the exampleused by Burstall and Darlington (1977) A foldunfold proof of the theorem lsquoif an elementoccurs in the concatenation of two lists if must exist in one of the lists independentlyrsquo isshown in Listing A2 in Appendix A
Definition The Definition rule (also known as Eureka) is used to introduce new functiondefinitions into the derivation One should be able to introduce new definitions at anytime not just the initial loading of the program Lines 1 to 5 of Listing A1 and 1 to 17 ofListing A2 demonstrate the use of the definition law
Instantiation Instantiation replaces a variable in a function definition by specific aconstructor patterns In Haskell 98 instantiation can be expressed in the arguments offunction definitions through pattern matching As our normalised form suppresses thisand our syntax does not support the construct instantiation will instead be expressed incase alternatives
In f-lite syntax instantiation of an argument vi with a constructor patterns c0 vs0 cm vsm in function f vs = x can be done by inserting a case expression at the root ofthe definition
case vi of c0 vs0 rarr x [ vi c0 vs0 ] cm vsm rarr x [ vi cm vsm ]
Then immediately perform the case of constructor transformation (caseOfCons inFigure 312) to propagate the pattern matching Lines 6 to 13 of Listing A1 illustrate theuse of the instantiation law
36
34 The FoldUnfold Laws
power4 x = times ( t imes x x ) ( t imes x x ) minus A b s t r a c t lsquo t i m e s x x lsquo minus power4 x = l e t sqr = times x x in t imes sqr sqr
i n c _ p a i r x y = Pai r ( ( + ) 1 x ) ( ( + ) 1 y )minus A b s t r a c t lsquo ( + ) 1 lsquo minus i n c _ p a i r x y = l e t inc = ( + ) 1 in Pair ( inc x ) ( inc y )
Listing 31 Examples of abstraction in f-lite
Residuating the expressions in the alternatives can improve the readability of a deriva-tion Listing A2 illustrates this style
In a non-strict language perp arguments should be considered in addition to the rangeof constructed values Failure to do so can lead to a derived function exhibiting differentbehaviour
Reduction of expressions containing perp proceeds as follows case expressions withstrict patterns and perp as the subject simplify to perp and applications of primitive functionswith perp as an argument simplify to perp Instantiation with perp is used in Listing A2 lines19 to 29
Unfolding Unfolding (or Inlining) has already been defined in Subsection 331 Weensure that sharing is preserved by providing the arguments to the inlined expressionthrough local variables rather than directly substituting the expressions
Folding Folding has already been defined in Subsection 332 There are no practicaldifferences between that definition and the one used in Burstall and Darlington (1977)
Abstraction Abstraction introduces a local definition for similar subexpressions topromote sharing In Haskell 98 local definitions can be functions of arbitrary aritySubexpressions can be generalised to factor out similar computation and reduce redund-ancy
All f-lite bindings must be unary variables These variables may contain partialapplications of functions but no new functions can be defined This restriction meansthat only equal expressions can be abstracted not similar computations This type oftransformation is known as common subexpression elimination
However this restriction does make it much easier to determine when an abstractionshould occur Listing 31 illustrates the application of these laws with examples
Laws One expression can be replaced by another known to be equivalent Equivalentscan be inferred from known theorems such as associativity and commutativity or maybe determined by previous derivations
37
3 Problem Analysis
f vs = v f vs = case v of alts
f vs = c xs f vs = case c xs of alts
f vs = f p xs f vs = case f p xs of alts
Figure 39 Root expressions that will cause simple termination
35 Supercompilation Strategy
Mitchell (2008 chap 4) showed that supercompilation (Turchin 1979) can lead to largeperformance boosts in a non-strict functional language On a particular example theword counting program performance even bested the equivalent program written in C
Supercompilation produces a chain of derivations by inlining function applicationsand simplifying the result This process is known as driving If this new functiondefinition embeds a previously derived result then driving terminates and the childrenof the outermost expression in the function body are residuated The residuals are thensupercompiled
In a little more detail the supercompilation strategy is as follows For some function
1 Check whether this function meets the termination criteria see Subsection 351
2 If it does not meet the termination criteria then drive
a) Inline a function application (see Subsection 352) and
b) Simplify the function (Subsection 353)
c) Continue supercompiling this new definition
3 If it meets the termination criteria then (Subsection 354)
a) If the driving terminated because of an embedding generalise the expressionwith respect to the function body it embeds
b) Tie expression If any of the expressionrsquos children are instances of a previouslyderived residual body then fold back into this definition Otherwise residuatethe child and supercompile the residual
4 When every residual has been supercompiled inline (Subsection 355) non-recursiveresidual definitions to alleviate unnecessary function application overheads
351 Termination Criteria
Two types of termination criterion are used for Mitchellrsquos (2008) supercompiler Thesimple termination criterion stops driving through a function if supercompiling the root ofthe function definition can not pass any more information to the others The definition ofthe simple termination criterion for f-lite is shown in Figure 39
A classic example of this is where the root of the definition is a case expression withan unbound variable as the subject As this case expression cannot be resolved one mayas well supercompile its children expressions separately (see Subsection 354)
38
35 Supercompilation Strategy
dive(x y) or couple(x y)homeo(x y)
exist c children(y) middot homeo(x c)dive(x y)
x sim y and forall(xc yc) zip(children(x) children(y)) middot homeo(xc yc)couple(x y)
Figure 310 The homeomorphic embedding relation homeo or E
f sim f same function namec sim c same constructor namev sim v same variable name
vL sim wL both are localx xs sim y ys length xs = length ys
let bsx in x sim let bsy in xy both bind the same variables
case x of altsx sim case y of altsy both contain the same patterns
Figure 311 The expression similarity relation sim
Our definition of the simple termination criterion differs slightly from Mitchell (2008chap 4) Our language includes lsquoprimitiversquo functions that cannot be inlined as they arehandled at hardware level To the supercomplilation strategy they behave very similar toa free variable Therefore a derivation containing a primitive function application at itsroot or a case expression on a primitive application should be residuated
The other termination criterion is if a previously derived definition is homeomorphicallyembedded in the current expression We say that x is a homeomorphic embedding of y if the relation homeo(x y) holds as described in Figure 310
352 Unfolding Selection
Functions are inlined using the lsquosharing preservingrsquo approach described in Subsec-tion 331 In a given derivation functions are inlined in order of evaluation and if theywill not cause driving to terminate If all inlinings will cause driving to terminate thenselect the first in the order of evaluation Otherwise terminate driving
353 Simplification Transformations for f-lite
Mitchell (2008) presents thirteen simplification laws used as part of the supercompilationstrategy He notes that while ldquosome of the rules duplicate code none duplicate work Allthe rules preserve the semantics and the sharing behaviour of an expressionrdquo
The simplification rules presented in Figure 312 are defined similarly to those that
39
3 Problem Analysis
( x xs ) ys (appOfApp)=rArr
x append( xs ys )
case c x1 xn of c v1 vn rarr y (caseOfCons)=rArr
let v1 = x1 vn = xn in y
( case x of p1 rarr y1 pn rarr yn ) zs (appToCase)=rArr
case x of p1 rarr y1 zs pn rarr yn zs
( let v1 = x1 vn = xn in y ) zs (appToLet)=rArr
let v1 = x1 vn = xn in y zs
case ( let bs in y ) of as (caseOfLet)=rArr
let bs in ( case y of as )
case ( case x of p1 rarr y1 pn rarr yn ) of as (caseOfCase)=rArr
case x of p1 rarr case y1 of as pn rarr case yn of as
case v of c vs rarr y (substituteVar)=rArr
case v of c vs rarr y [ v c vs ]
Figure 312 Simplification laws for f-lite
40
35 Supercompilation Strategy
let v1 = x1 vi = xi vn = xn (letInCase)in ( case y of pj rarr yj )
=rArrlet v1 = x1 vn = xn
in ( case y of pj rarr let vi = vj in yj )
where vi 6isin f ree(y) cup⋃j f ree(xj)
let v1 = x1 vi = xi vn = xn in y (inlineLet)=rArr
( let v1 = x1 vn = xn in y ) [vixi]
where vi 6isin⋃
j f ree(xj) and only occurs once in f ree(y) or xi is a variable
let vi = c xi1 xim in y (splitLet)=rArr
( let
vFi1 = xi1
vFim = xim
in y )[ vi c vFi1 vF
im ]
where vFi1 vF
im are fresh variables
let in x =rArr x (removeLet)
let bs1 in ( let bs2 in x ) (letInLet)=rArr
let ( bs1 ++ bs2 ) in x
where no variables names bound in bs2 occur in bs1
Figure 313 New and augmented simplification transformations for f-lite
41
3 Problem Analysis
Mitchell uses for his core language However our core language has differs from thatused in Mitchell (2008) as explained in Section 32 For example we allow multiplebindings within a single let expression Laws that required major alterations are definedin Figure 313
For letInCase only bindings that are not used in the either the subject of the caseexpression nor any other binding that does not meet this criteria Any non-inlinablebindings must be retained in the inlineLet law Furthermore the law stating that bindingswhere the expression is itself is a variable can be inlined is merged in Finally the lawsplitLet needs to ensure that the variable is substituted into any referencing bindings
To tidy up any lsquoneedlessrsquo let expressions two new rules are introduced The remove-Let remove merely replaces a let containing no bindings with the subject expressionThe letInLet rule coalesces lets in much the same was as the appOfApp rule does withapplications
The f-lite language does not include anonymous lambda abstractions Lambda are notrequired for any of the simplification rules The lam-app law converts an applied lambdainto a let expression so can be safely omitted from our collection
These simplification laws can be non-deterministically applied throughout the abstractsyntax tree until it reaches a normalised simplified form
354 Residuation Folding and Generalisation
When driving terminates it is necessary to produce residual definitions so that supercom-pilation can continue The production of these residual expressions depends on a varietyof conditions
Most specific generalisation If compilation terminates due to a homeomorphic embed-deding one may be able to generalise the current definition with respect to the similarfunction We utilise the most specific generalisation techniques described by Soslashrensen et al(2008)
If y is the candidate function body and x is the expression previous residual that wasfound to be homeomorphically embedded in y msg(x y) is the result of applying therewrite rule in Figure 314 to the triple (vF vF = x vF = y) A fresh variable isrepresented by vF The rewrite rule is applied until no further rewrites are possible
tg
vF = σ(x1 xn) cup θx
vF = σ(y1 yn) cup θy
rarr
tg[vFσ(vF1 vF
n)]
vF1 = x1 vF
n = xn) cup θx
vF1 = y1 vF
n = yn) cup θy
Figure 314 Most specific generalisation rewrite rule where v1 vn are fresh variables
and the σ( cs ) detonates an expression spine and its children
If most specific generalisation returns (xprime bsx bsy ) = msg(x y) the generalisedexpression is let bsx in xprime This is only valid if bsx contains no free variables as any
42
36 Requirements A Transformation Framework
subsequently bound variables will be references before their binding
Mitchellrsquos generalisation Mitchell (2008 chap 4) extends this method to deal with thislimitation His generalisation denoted by x y utilises lambda expressions to lift freevariables to the root level As is able to factor out more expressions than most specificgeneralisation subsequent derivations may be able to tie back sooner Our core languagedoes not include anonymous lambda abstractions so we cannot immediately make use ofthis technique
Residuation and Folding Whether or not generalisation is feasible the children of theroot expression are residuated under certain conditions
If the child contains no inlinable function applications there is no reason to produce aresidual so no residuation occurs
If the child is an instance of to a previously derived function body it can be foldedback into an application of that function This process is discussed in Subsection 332
Otherwise a residual expression is produced and supercompilation continues
355 Inlining Unnecessary Residuals
Supercompilation terminates when every residual has been processed Residuationcreates many function applications that would add overhead if they were to be executedin this form
Many of these residuals can be inlined back after supercompilation into their ori-ginating expression We inline application to a residual that is not recursive across oursupercompiled functions As our inline transformation uses let expressions to preservesharing we then apply the inlineLet and removeLet simplifications (see Figure 312) toremove any redundant abstraction
36 Requirements A Transformation Framework
Our analysis so far has indicated several requirements to be able to perform transforma-tions such as foldunfold and supercompilation One requires a method for specifyingtransformations in a manner similar to that defined in Section 31 Basic transforma-tion functions (Subsection 321) and primitive transformations (Section 33) need to beprovided so that transformations and transformation strategies can be constructed usingthem Throughout the analysis there have been warnings about variable capture and theuse of fresh variables has been highlighted The transformation framework must takethese into account
Existing systems Existing program transformation systems were investigated in Chapter 2Common themes included (a) the ability to control the tree traversal strategy (b) thepotential to transform arbitrary languages (c) a facility by which one could introducecustom transformations to the system Interactive program transformation tools inparticular needed to provide an intuitive user interface suitable for ldquoany novice functionalprogrammerrdquo
43
3 Problem Analysis
Arbitrary languages Supercompilation appears to be particularly well suited to call-by-need functional languages This project is orientated towards work on a particularplatform so it seems unnecessary to provide full flexibility in the types of languages thatcan be operated upon
However it is convenient to abstract transformation operations away from the actualtarget language as far as is practical to so It would be unfortunate to have to completelyre-implement the system if changes to the f-lite specification occur
History and annotation Whether transformations are being applied interactively or auto-matically it is useful to maintain a history of previous derivations for later analysis Fora strategy like supercompilation it is essential for the control of residuation
The annotation of these derivations with information about what transformationswere performed to produce the new result allows the debugging of transformations andgives greater insights into the path of derivation in proof and optimisation
To this end two additional low-level operations should be supplied One is to introducean annotation attached to the current instance In the f-lite language annotations canbe introduced as comments at declaration level Another operation is to clone a currentinstance and its annotations as a historical definition As only the first function definitionis consider the lsquoactiversquo definition historical definitions can be stored directly below it inthe abstract syntax tree
Automatic vs Interactivity A full automatic program transformer merely requires anoriginal program and a strategy to perform over the abstract syntax tree For human-directed program transformation the tool needs to expose information about whichtransformations are applicable given certain states and should give the user the ability totarget specific expressions
Systems found in literature such as CIP (Bauer 1979) and ZAP Feather (1982) usedcontrol languages entered at a command-line to instruct the tool These are very much aproduct of their age Given todayrsquos profusion of graphical user interfaces for softwaredevelopment a minimum requirement would be a cursor and dialogue based interface
The output from an interactive transformer should be lsquopleasantrsquo for the human to readThe rules for generating fresh variable names should be derived with care and programcode should be formatted according to language style guidelines When displayed formanipulation syntax highlighting would ease the usersrsquo interactions
37 Summary
We have presented a notation for describing transformations (Section 31) and definedthe preliminary lsquobuilding blocksrsquo of transformations (Subsection 321) A core languagefor transformed programs has been defined (Section 32) a normalised form of theReduceron 2rsquos f-lite taking advantage of already implemented transformations
Using these as a foundation we have described a variety of transformations from theprimitives in Section 33 to the complex supercompilation strategy in Section 35 Theissues involved in applying supercompilation to the f-lite language have been emphasisedand potential solutions have been identified
44
37 Summary
Finally drawing on the background provided by the literature and the prerequisitesof transformation that have become apparent through the course of the Problem Ana-lysis requirements have been extracted for the transformation tool These are listed inFigure 315
The interactive transformation system requires
1 to accept any program written in our core language normalised f-lite as input butbe flexible enough to handle changes in language specification without requiringsignificant revising
2 a format for describing transformations and strategies like those in Section 33Section 34 and Section 35 preferably in a form similar to that defined in Section 31
3 to display the current state of the programs abstract syntax tree in a human-friendlyformat following standard style guidelines
4 to permit specific subexpression to be targeted for transformation and to informthe user which transformations are applicable
5 to maintain a historical trace of previously derived results and to document howresults were derived both for the userrsquos information and to support complexstrategies decision making processes
6 to output to file the final state of the transformed program in human-readable andreadily executable forms
Figure 315 Requirements for Optimus Prime
45
4 Design and Implementation
We develop the requirements and concepts described in the previous chapter into aworking system dubbed Optimus Prime The implementation can be be broken downinto a number of discrete lsquotasksrsquo illustrated by Figure 41
Parser Zipper Conversion Control
Strategy
Transformations
Pretty Printer
Display Output
Literate Output
Tidy Output
SimplificationTransformations Termination Unfold
Generalisation Residuate
Supercompilation
Figure 41 Optimus Prime architecture
The parser and zipper conversion (discussed in Section 41) read an f-lite source fileinto Optimus Primersquos internal syntax representation Transformations are implementedusing the techniques described in Section 42 and Section 43 and strategy specifics arehighlighted in Section 44 Interfaces for controlling the the transformation process andproducing pretty printed output are discussed in Section 45
The implementation specifics for implementing FoldUnfold and Supercompilationare presented in Section 44
41 Representation through Syntactic Zippers
This project is essentially concerned with the representation and manipulation of programsyntax We need a suitable data structure It must be easily manipulated and it musthold the information required to perform transformations
Simple trees Initially a typed tree structure similar to that described in Listing 213 wasconsidered to represent the abstract syntax tree It would be advantageous to implementnew transformations in a similar manner to Naylorrsquos existing optimisations Further-more as Naylorrsquos technique is inspired by Mitchell and Runciman (2007) Uniplate theSupercompilation transformations described by Mitchell (2008) could be easily mappedonto that model
46
41 Representation through Syntactic Zippers
minusminus Zi ppe r p o s i t i o ntype ZipPath = [ I n t ]
minusminus S y n t a c t i c z i p p e r nodedata ZipParent a = N a ( ZipParent a ) minusminus I n v e r t a b l e Tre e Node
| H minusminus Empty Tre e Node
minusminus S y n t a c t i c z i p p e r s t r u c t u r etype Zip a = ( ZipPath ZipParent a BindingMap ( ZipParent a ) )
minusminus Type c l a s s o f a z i p p a b l e syn ta x s t r u c t u r ec l a s s Zippable a where
minusminus Swap a z i p p e r node with a t r e e nodes c h i l dhole i fy rsquo I n t rarr ZipParent a rarr a rarr ( a ZipParent a )
minusminus Number o f c h i l d r e n a t r e e node hasexpWidth rsquo a rarr I n t
minusminus Add an e x p r e s s i o n s b i n d i n g s t o t h e s c o p eaddBindings I n t rarr ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus Remove an e x p r e s s i o n s b i n d i n g s from t h e s c o p eremoveBindings ZipParent a rarr
BindingMap ( ZipParent a ) rarrBindingMap ( ZipParent a )
minusminus I n i t i a l i s e t h e v a r i a b l e mapi n i t F r e s h Zip a rarr VarMap
Listing 41 Syntactic zipper types
Huetrsquos Zipper The issues is that in a purely functional language data structures arenot mutable ldquoFor trees this amounts to modifying an occurrence in a tree non-destructively bycopying its path from the root of the treerdquo (Huet 1997) Non-mutable tree structures are notwell suitable for editor buffers states Huet (1997) Any modification requires traversaldown the tree to the appropriate point and the reconstruction of the tree back to the root
Instead he suggests a zipper a tree ldquoturned inside-out like a returned gloverdquo The structureholds a description of the current position the tree from the current position down andan inverted tree back to the root
Syntactic Zipper Our interpretation of a zipper differs from Huetrsquos An attempt wasmade to generalise the concept to any abstract syntax tree definition Our zipper (definedin Listing 41 and visualised in Figure 42) contains an integer list representation of thepath to the current position in the tree and a ZipParent structure The ZipParent structurecontains a syntax node and either a terminating symbol or another ZipParent structuredescribing the path back to the tree root
Any traversal or update operation (see Listing 42) on a Zipper structure should
47
4 Design and Implementation
A
B C D
E F
G
A
B D
E
F
GC
[ 0 1 ][ ]
Figure 42 Example of a zipper structure The left shows a zipper at the root of the treeThe right illustrates the zipper after being navigated to the first child of thesecond child of the root
f = 1g f = let h = f in let f = 2 in f
Figure 43 Example of scoping issues
maintain the following invariants (in terms of the data types in Listing 41)
bull For any ZipParent N e ( N eprime pprime ) (a context) exactly one child of eprime is H This childis the placeholder for the subject e
bull For any other non-context ZipParent no children may be H No placeholders arerequired
bull In a Zipper ( zp N e p bm ) zp = [ ] if and only if p = H Both of theseconditions imply that the zipper is positioned at the root of the tree
bull Each element of zp relates to the position of the placeholder child in each contextTherefore the length of zp is equal to the distance moved down the tree
Scoping for Zippers Section 32 highlighted that f-lite allows variable names to be reusedin separate scopes For example in Figure 43 the variable f is defined three timesThere is a top-level definition of f the function g uses f as an argument and f isused as a local variable
One must ensure that a variable name is referring to the appropriate definition Thefirst use of f (in the first let) refers to the argument of g but the second refers to the fdefined in the second let
48
41 Representation through Syntactic Zippers
minusminus B u i l d a z i p p e r out o f a z i p p a b l e t r e ei n i t Z i p Zippable a rArr a rarr Zip a
minusminus Move t o p a r e n t o f t h e c u r r e n t nodemoveUp Zippable a rArr Zip a rarr Zip a
minusminus Move t o a c h i l d r e n o f t h e c u r r e n t nodemoveDown Zippable a rArr I n t rarr Zip a rarr Zip a
minusminus Move t o t h e r o o tmoveTop Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e p r e c e d i n g s i b l i n gmoveLeft Zippable a rArr Zip a rarr Zip a
minusminus Move t o t h e s u c c e e d i n g s i b l i n gmoveRight Zippable a rArr Zip a rarr Zip a
minusminus I s t h i s t h e r i g h t m o s t s i b l i n g isEdge Zippable a rArr Zip a rarr Bool
minusminus Does t h i s node have any c h i l d r e n i sTerminal Zippable a rArr Zip a rarr Bool
minusminus R e p l a c e t h e e x p r e s s i o n in t h e c u r r e n t nodeupdate Zippable a rArr a rarr Zip a rarr Zip a
minusminus Apply a f u n c t i o n t o t h e c u r r e n t nodeupdateWith Zippable a rArr ( Zip a rarr a ) rarr Zip a rarr Zip a
minusminus Apply a monadic f u n c t i o n t o t h e c u r r e n t nodeupdateWithM ( Zippable a Monad m) rArr ( Zip a rarrm a ) rarr
Zip a rarrm ( Zip a )
Listing 42 Primitive syntactic zipper operations
49
4 Design and Implementation
import qual i f ied Data Map as Map
minusminus R e p r e s e n t a t i o n o f v a r i a b l e b i n d i n g stype BindingMap a = MapMap S t r i n g [ VariableType a ]
minusminus V a r i a b l e b i n d i n gdata VariableType a = Global I n t [ S t r i n g ] a minusminus G l o b a l f u n c t i o n
| Known I n t a minusminus L o c a l d e f i n i t i o n| Unknown I n t minusminus Func t i on argument| S u b s t i t u t i o n a minusminus S u b s t i t u t e f o r e x p r e s s i o n
Listing 43 BindingMap data structure
This ability to reuse variable names in separate scopes is a common feature of program-ming languages Several options for handling scoping issues were considered Theseincluded (a) only allowing variable names to be used once for the entire program notjust within scopes (b) traversing back up the tree to find the nearest appropriate binding(c) storing the depth of the appropriate binding with the variable reference
Option (a) was discounted for making code too awkward to write and as a result lessreadable on output Option (b) was considered too expensive an operation at the point ofus Option (c) seemed to require far too much computation on a manipulation of the tree
Instead we abstract the concept of scopes to a structure known as a binding map(Listing 43) The binding map maintains a stack of definitions for each variable name ata given point in the zipper Any traversal or update operation on a zipper must ensurethat the binding map is maintained
Zipper f-lite and Parsing The syntax described in Section 32 is represented by the zipperstructure defined in Listing 44 Naylorrsquos own parser is used to read in f-lite source filesThe parser uses the data type defined Listing 213 This representation is then convertedinto the zipper type for f-lite This approach reduces development time and ensures thatno major deviations from Naylorrsquos f-lite are made
42 Zipper Query and Modification
Primitive operations for moving through a zipper structure were shown in Listing 42However more abstract query and modification functions are required for the specifica-tion and application of transformation strategies
Similar to uniplate our query operations (Listing 45) produce lists of zipper nodesfitting a relationship with the input node These lists can be queried further through listcomprehensions to select nodes meeting specific requirements For example the functionappPaths (as defined in Listing 46) returns the list of ZipPaths where applicationsoccur in the descendants of z
Modifications can be propagated through a zipper using the modification operationspresented in Listing 47 These apply a partial transformation function to appropriatenodes in the tree Where a transformation is not applicable to a certain structure no
50
42 Zipper Query and Modification
type ZipFLite = Zip FLi te
type Id = S t r i n g
type Binding = ( Id ZipParent FLi te )
data FLi te = Prog [ ZipParent FLi te ]| Decl Id [ Id ] ( ZipParent FLi te )| App ( ZipParent FLi te ) [ ZipParent FLi te ]| Let [ Binding ] ( ZipParent FLi te )| Case ( ZipParent FLi te ) [ ZipParent FLi te ]| Alt ( ZipParent FLi te ) ( ZipParent FLi te )| Var Id| Con Id| I n t I n t| Annotations [ S t r i n g ]| S e l e c t e d ( ZipParent FLi te ) minusminus Used f o r d i s p l a y
instance Zippable FLi te where
Listing 44 Zipper structure for f-lite
minusminus A l i s t o f a nodes c h i l d r e nch i ldren Zippable a rArr Zip a rarr [ Zip a ]ch i ldren x( _ e _ ) = map ( f l i p moveDown x ) [0 ( expWidth e minus 1 ) ]
minusminus A l i s t o f a nodes d e c e n d a n t s preminuso r d e r t r a v e r s a ldecendants Zippable a rArr Zip a rarr [ Zip a ]decendants x = concat [ c decendants c | c ltminus ch i ldren x ]
minusminus A l i s t o f a l l nodes in a s u b t r e e preminuso r d e r t r a v e r s a lsubtree Zippable a rArr Zip a rarr [ Zip a ]subtree x = x decendants x
Listing 45 Zipper query functions
appPaths Zip FLi te rarr [ ZipPath ]appPaths z = [ zp | ( zp N (App _ _ ) _ _ ) ltminus decendants z ]
Listing 46 Example of zipper query using list comprehensions
51
4 Design and Implementation
minusminus Apply a t r a n s f o r m a t i o n t o e a c h c h i l dupdateChildren Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e preminuso r d e r t r a v e r s a lupdateSubtree Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply a t r a n s f o r m a t i o n t o a s u b t r e e pos tminuso r d e r t r a v e r s a lupdateSubtreeB Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Apply preminuso r d e r but f a i l i f no m o d i f i c a t i o n i s madeupdateSubtreeF Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus u p d a t e S u b t r e e F u n t i l no more m o d i f i c a t i o n s can be madenormalise Zippable a rArr ( Zip a rarr Fresh a ) rarr
Zip a rarr Fresh ( Zip a )
minusminus Normal i se k e e p i n g n o t e o f what t r a n s f o r m a t i o n s a r e a p p l i e dnormalises Zippable a rArr [ ( S t r ing Zip a rarr Fresh a ) ] rarr
Zip a rarr Fresh ( [ S t r i n g ] Zip a )
Listing 47 Zipper modification functions
change is made However updateSubtreeF is undefined when no modifications aremade to any node in the subtree
The normalise function applies a transformation throughout a tree repeatedly untilno more applications are possible In the variant normalises a list of transformationsand names are supplied An accumulator holds a list of all the transformations that havebeen applied during the normalisation process
Discussion Several traversal techniques and libraries exist for querying and updatingimmutable trees Mitchell and Runciman (2007) present a library for ldquogeneric traversalsover recursive data structuresrdquo by alleviating the need for so called boilerplate code
Listing 48 shows boilerplate for performing query and modification operations ona simple tree structure Much of the code is repeated for different pattern expressionsMitchellrsquos Uniplate library removes the mundane code by abstracting all query andmodification operations to use a single uniplate function
An instance of Uniplate typeclass is written for each structure by implementing theuniplate function An example Uniplate instance is shown in Listing 49 along withUniplate implementations of the consts and vars functions
Our Syntactic Zipper data structure replicates this functionality Similar to Uniplatean instance is written for the Zippable typeclass However an intermediary typeZipParent must be inserted into the tree structure This is to handle the lsquoholesrsquo leftwhen child is moved to be the subject of the zipper Listing 410 illustrates the adjustedform of Logic named LogicZ and its corresponding Zippable instance
52
42 Zipper Query and Modification
data Logic = And Logic Logic| Or Logic Logic| Not Logic| Var iab le S t r i n g| Constant Bool
cons ts Logic rarr [ Bool ]cons ts (And x y ) = cons ts x ++ consts ycons ts ( Or x y ) = cons ts x ++ consts ycons ts ( Not x ) = cons ts xcons ts ( Var iab le _ ) = [ ]cons ts ( Constant x ) = [ x ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs (And x y ) = And ( vars bs x ) ( vars bs y )vars bs ( Or x y ) = Or ( vars bs x ) ( vars bs y )vars bs ( Not x ) = Not ( vars bs x )vars bs ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
| otherwise = Var iab le xwhere
b = lookup x bsvars bs ( Constant c ) = Constant c
Listing 48 Example of boilerplate code
instance Uniplate Logic whereuni p la t e (And x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr And x rsquo y rsquo )un i p la t e ( Or x y ) = ( Two (One x ) (One y )
(Two (One x rsquo ) (One y rsquo ) ) rarr Or x rsquo y rsquo )un i p la t e ( Not x ) = ( One x One x rsquo rarr Not x rsquo )un i p la t e x = ( Zero Zero rarr x )
cons ts Logic rarr [ Bool ]cons ts l = [ x | Constant x ltminus universe l ]
vars [ ( S t r ing Bool ) ] rarr Logic rarr Logicvars bs l = transform vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = x
Listing 49 Uniplate representation of Listing 48
53
4 Design and Implementation
data LogicZ = And ( ZipParent LogicZ ) ( ZipParent LogicZ )| Or ( ZipParent LogicZ ) ( ZipParent LogicZ )| Not ( ZipParent LogicZ )| Var iab le S t r i n g| Constant Bool
instance Zippable LogicZ wherehole i fy rsquo 0 h (And x y ) = (And h y x )ho le i fy rsquo 1 h (And x y ) = (And x h y )ho le i fy rsquo 0 h ( Or x y ) = ( Or h y x )ho le i fy rsquo 1 h ( Or x y ) = ( Or x h y )ho le i fy rsquo 0 h ( Not x ) = ( Not h x )
expWidth rsquo ( Var iab le _ ) = 0
expWidth rsquo ( Constant _ ) = 0
expWidth rsquo ( Not _ ) = 1
expWidth rsquo _ = 2
addBinding _ _ = idremoveBinding _ _ = id
cons ts Zip LogicZ rarr [ Bool ]cons ts l = [ x | ( _ N ( Constant x ) _ _ ) ltminus subtree l ]
vars [ ( S t r ing Bool ) ] rarr Zip LogicZ rarr Zip LogicZvars bs l = updateSubtree vars rsquo l
wherevars rsquo ( Var iab le x ) | i s J u s t b = Constant ( fromJust b )
whereb = lookup x bs
vars rsquo x = die
Listing 410 Our zipper representation of Listing 48
54
43 Transformation Specification
emptyLet Zip FLi te rarr Zip FLi teemptyLet ( zp N ( Let [ ] (N e H) ) p bm) = ( zp N e p bm)emptyLet z = z
Listing 411 Complex empty let removal for f-lite
Syntactic Zipper versions of consts and vars are provided in Listing 410 Noticethe close similarity of definitions for Uniplate in Listing 49 Our zipper operations alsoprovide additional information that Uniplate cannot such as the path to the returnednode the context of the node and the scope state at this point of the tree From a singlequeried node one could for instance move to one of its siblings
However Uniplatersquos unawareness of these means that it requires less memory and isless computationally intensive for the same query and modification operations A fullperformance comparison can be found in Section 51
Uniplate and the Syntactic Zipper only allow homogenous type traversals Mitchellextends his concept to performing heterogeneous traversals with Biplate Biplate requiresmulti-parameter type classes an extension to Haskell 98
The CLASE cursor library (Allwood and Eisenbach 2008) provides similar multi-typetraversal within a zipper structure However their technique requires even more Haskell98 extensions than Biplate including generalised abstract data types type familiesrank-N types and mutli-parameter type classes
The Syntactic Zipper library has the advantage of only requiring pure Haskell 98 Thisis at the expense of heterogeneous data types like Naylorrsquos f-lite abstract syntax tree inListing 213 We lose the ability to use type checking to ensure that syntactic categoriesoccupy the correct places It is up to the developer to ensure their transformations do notbreak the programrsquos syntax
43 Transformation Specification
Transformations were specified in Section 31 In terms of the structures that have beenintroduced so far a transformation maps one syntactic zipper to another
Zipper unsafe transformation A Haskell function can be defined such it only patternmatches on applicable expressions Where a transformation is not appropriate thefunction returns the zipper unmodified Listing 411 illustrates a transformation writtenin this style
Zipper safe transformation A transformation should only alter the structure of the cur-rent zipper expression refraining from modifying the ancestry and leaving the zipperin the same position The binding map should be updated to reflect any effects themodification of the expression has on the current scope
A transformation may still require the use of this contextual information while notmodifying it directly The updateWith function (type signature defined in Listing 42)modifies a zipper using a function that takes the full zipper as its input but only returns
55
4 Design and Implementation
emptyLet Zip FLi te rarr FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = eemptyLet ( _ N e _ _ ) = e
Listing 412 Simpler empty let removal for f-lite
emptyLet Zip FLi te rarr Maybe FLi teemptyLet ( _ N ( Let [ ] (N e H) ) _ _ ) = J u s t eemptyLet _ = Nothing
Listing 413 Monadic empty let removal for f-lite
the new sub-expression structure This ensures that the criteria above are met Listing 412
represents the same transformation written for the updateWith function
Applicable transformation So far when a transformation is not appropriate for a partic-ular expression the expression is returned unmodified However a strategy may requirethe knowledge of whether a transformation has been applied or not A transformationcould be defined as a partial function of zippers to expressions
Haskell allows the encoding a partial function using the Maybe monad Where theoutput of the function is undefined it returns Nothing Otherwise the function returnsJust the value The updateWithM operation lifts the partial computation results to thewhole zipper Listing 413 provides an example of our example transformation rewrittenfor this format
Fresh variable supply Several operations and transformations described in Chapter 3
require the production of fresh variable namesOur solution is to use a variable map as defined by in Listing 414 A map is maintained
that holds indexes for variable prefixes When a new fresh variable name is requestedusing the getFresh function the index for that variable prefix will be incremented
The advantage of using a map of variable prefixes is that fresh variable names can berequested that are similar to an existing variable name For example when lsquofresheningrsquo thevariable names in a function body before inlining fresh variables can be selected so thatthe function still reads similarly to the original definition
This map could have been held inside the Syntactic Zipper as it already holds otherinformation about the abstract syntax tree However so far only information that isupdated on tree traversal (the zipper position subject context and scope) have beenincluded and it would seem out of improper to place a structure that is constant regardlessof tree position
Instead a State monad is used to encapsulate the current state of the variable mapThe State monad can be used to alleviate the need to pass around the state explicitlywhile still fitting the functional model
The State monad is coupled with the Maybe represented by using a monadic trans-former to form the Fresh monad Both the state and result can be backtracked when atransformation is not appropriate As this composite Monad is an instance of the Monadtypeclass it can still operate with the updateWithM function as above
56
44 Transformation Specifics
import qual i f ied Data Map as Mapimport Data Char
type VarMap = MapMap S t r i n g I n t
type Fresh a = Sta teT VarMap Maybe a
splitName S t r i n g rarr ( S tr ing I n t )splitName xs = ( reverse s_ i f n u l l n_ then 0 e lse ( read reverse ) n_ )
where( n_ s_ ) = ( span i s D i g i t reverse ) xs
getFresh S t r i n g rarr Fresh S t r i n ggetFresh s = l e t ( s rsquo _ ) = splitName s in do
vs ltminus getn ltminus re turn (Map f indWithDefault 0 s rsquo vs + 1 )put (Map i n s e r t s rsquo n vs )re turn ( s rsquo ++ show n )
Listing 414 VariableMap state
44 Transformation Specifics
441 Low-Level Definitions
In Subsection 321 several preliminary meta-syntactic functions were abstractly definedSome such as relationships between expressions are implemented directly by thetraversal methods described in Section 42
Others require concrete Haskell definitions so that the transformations defined in Sec-tion 33 Section 34 and Section 35 can be implemented in similar fashions Listing 415
outlines the type signatures for these functions
Two versions of the lsquofree variables of an expressionrsquo function are defined One thatonly lists each variable name once for any number of free occurrences in an expres-sion (freeSet) and one that lists a variable name each time there is a free reference(freeMulti)
The substitution function subs is target language specific The usage shown inListing 415 is not far removed from the notation used throughout the previous chapterThe tree is traversed in a bottom-up manner and any number of substitutions can bemade in one application of the substitution function
The functions length and zip are already implemented by the Haskell 98 preludeThe (++) combinator performs the same general function as append but we use theopportunity to define one that produces required by transformation
The transformation documentation functions clone and annotate (as required bySection 36) are f-lite specific like subs
57
4 Design and Implementation
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n nod u p l i c a t e s
f r e e S e t Zip a rarr [ S t r i n g ]
minusminus L i s t s t h e names o f any f r e e v a r i a b l e in t h e e x p r e s s i o n i n c l u d e sm u l t i p l e r e f e r e n c e s
freeBag Zip a rarr [ S t r i n g ]
minusminus S u b s t i t u t i o n where x lsquo subs lsquo [ ( v y ) ] = x [ v y ]subs Zip FLi te rarr [ Binding ] rarr Zip FLi te
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d elength [ a ] rarr I n t
minusminus Combines two l i s t s o f z i p p e r s i n t o one l i s t o f e x p r e s s i o n sappend [ Zip a ] rarr [ Zip a ] rarr [ ZipParent a ]
minusminus I n c l u d e d in H a s k e l l 98 p r e l u d ezip [ a ] rarr [ b ] rarr [ ( a b ) ]
minusminus Clone t h e c u r r e n t f u n c t i o n d e f i n i t i o n i n t o t h e h i s t o r yclone Zip FLi te rarr Zip FLi te
minusminus Add a l i s t a n n o t a t i o n s t o t h e c u r r e n t l i s t a t t a c h t o a f u n c t i o nd e f i n i t i o n
annotate [ S t r i n g ] rarr Zip FLi te rarr Zip FLi te
Listing 415 Preliminary definitions
442 Transformation Rules and Strategies
The FoldUnfold laws are implemented as described in Section 34 using the techniquesmanner described in Section 43 and the low-level functions described above The lsquoeurekarsquolsquoinstantiationrsquo and lsquolawsrsquo rules require an command-line interface for the user to enterdefinitions and patterns The lsquoabstractionrsquo rule searches for common subexpressionwithin the currently selected expression Only the fold and unfold laws can operateautonomously
The supercompilation strategy (of Section 35) is similarly defined out of the trans-formation lsquobuilding blocksrsquo The simplification transformations are defined using thetechniques presented in Section 43 and the normalises modification function is usedto apply and annotate these transformations across a definition
An scUnfold transformation is defined over function declarations This transforma-tion inlines the first function application that upon inlining and simplification does notcause the termination criteria to be triggered Failing this the first function application isinlined The transformation is undefined when no function can be unfolded
The selective unfolding transformation and the simplifying normalisation are combinedto form the driving function When a termination criterion is triggered generalisationoccurs where appropriate and functions are residuated as described in Section 35
The driving process is then triggered on the residual functions until all residuals have
58
44 Transformation Specifics
Unfold an application
Simple Termination
Embedding
No
Apply simplifications
Generalise
Residuate supercompilable
children or replace with equivalent function
application
Residuals to supercompile
Yes
Start
No
Y
Y
Inline unnecessary
function applications
No
Stop
Figure 44 The supercompilation algorithm
59
4 Design and Implementation
been supercompiled Any non-recursive functions are inlined to reduce unnecessaryfunction application overhead Figure 44 depicts the algorithm as a flowchart
45 Output and Control
We have previously discussed how programs are parsed into the Syntactic Zipper rep-resentation (Section 41) However the state of the internal representation needs tobe returned to the user in an appropriate form Furthermore either the user or someother decision making mechanism need to guide the transformation process throughappropriate interfaces
451 Pretty printing
The abstract syntax tree needs to be returned to the user in the original programmingcode format We use the wl-pprint combinator library to produce output in the literateand core f-lite forms An extension to the wl-pprint library ansi-wl-pprint isused to produce syntax coloured output for display during interactive transformationBoth of these libraries are based on the paper lsquoA Prettier Printerrsquo by Wadler (1998) andare freely available in the Hackage repository
This library provide a large variety of combinators for concatenating lsquodocumentsrsquotogether To name a few possible operations documents can be combined with a spacebetween with a line-break between or more interestingly with a line-break only where aspace would cause the line to be too long
A maximum line width and a relative ribbon width are supplied to the algorithmThese determine how long a line can be and introduce line-breaks where appropriate toenforce these restrictions wherever possible
The appropriate use of these combinators allow us to produce program outputs thatconform to standard Haskell rsquo98 style guidelines making it easier for humans to read theresulting code
Additional functions in the ansi-wl-pprint library utilise ANSI escape sequencesto produce colours on a terminal supporting their display The functions allows thecolour coding of syntax when displaying programs in an interactive fashion Syntaxhighlighting makes interpreting programs a far more enjoyable experience for the user
Using these two closely related libraries three output formats can be generated for agiven f-lite abstract syntax tree
Display The display format is shown on-screen during interactive transformation usingthe ansi-wl-pprint Code is formatted in layout-sensitive Haskell style wherebraces and semicolons are omitted Colour-coded syntax highlight is used toimprove the usersrsquo experience A cursor is displayed for the currently selectedexpression Figure 45 shows the display output
Literate The literate format uses the Literate Haskell style to permit rich contextualinformation about transformation derivations The definitions are surround by LATEXcomments describing the transformation steps Current definitions are enclosedin code environments while previous derivations are in spec environments Thisallows a compiler to distinguish between them The lhs2TeX tool allows these
60
45 Output and Control
Figure 45 The interface of Optimus Prime
Literate Haskell files to be compiled into LATEX documents Appendix D shows anexample of the literate output
Tidy The tidy format is the most readily executable form Only the current definitionsare used and then only the definitions that are reachable from the main functionCode is formatted in the layout-insensitive explicit style as required by the f-litespecification The tidy format can be immediately executed by Naylorrsquos compilerand interpreter Listing E1 shows an example of tidy output
452 Interactive Control
Using the display output to view the current state of the Syntactic Zipper the usertraverses the abstract syntax tree uses in the arrow keys Right moves to a child expressionand left to the parent The down and up keys move through siblings The currently selectexpression is given a blue background
To perform a transformation the user presses the lsquoTrsquo key and is presented with a menuof permissible transformations for this expression An example of the interface in thisstate is shown in Figure 45
The current state of the Syntactic Zipper can be saved to a file using the lsquoLrsquo key forliterate output and lsquoSrsquo for tidy output
61
4 Design and Implementation
453 Autonomous Control
Autonomous control is created quite simply by applying encapsulating interactivelyactivated transformation strategy in a command-line tool Programs are passed in alongwith any strategy parameters through command-line arguments The program is thenreturned to stdout in either the tidy or literate formats described above
46 Summary
Optimus Prime is a tool that permits both interactive and automatic transformationson f-lite programs Transformations are specified as operations on the toolrsquos internalrepresentation of the abstract data structure the Syntactic Zipper All the requirementsnoted at the end of Chapter 3 have been fulfilled as follows
The structure operations on the structure and transformation types have been detailedSection 41 Section 42 and Section 43 These fulfil to Requirements 1 and 2 in Figure 315
The generic capabilities of the Syntactic Zipper library permit the f-lite language tobe extended without having to change specific traversal functions and most existingtransformations While it is slower at full tree traversal then other techniques it isable to support the concept of an interactive cursor through expressions So it fulfilsRequirements 1 and 4 in Figure 315
A Wadler (1998) style pretty printer is used to produce a variety of outputs for varyingneeds (Subsection 451) One of these outputs is used to display the abstract syntax treeduring interactive transformation and allows the user to move through the tree in anintuitive manner (Subsection 452) In this way we fulfil to Requirements 3 4 and 6 inFigure 315
Finally functions have been made available to store histories and annotate transforma-tion traces in accordance with to Requirement 5 of Figure 315
62
5 Results and Evaluation
This project has investigated techniques for performing interactive transformation andapplying the Supercompilation strategy to the f-lite language In this chapter we discussthe results obtained
The performance of the core component of Optimus Prime the Syntactic Zipperlibrary described in Section 41 and Section 42 is assessed in Section 51
The capabilities of the Optimus Prime Supercompiler for the f-lite language areevaluated in Section 52 Supercompilation times and other statistics are collected for thesupercompilation of a selection of programs The performance of these supercompiledprograms is then compared to the performance of the original programs
51 Performance of the Syntactic Zipper Library
In Section 42 the performance of the Syntactic Zipper library was discussed and brieflycompared with other tree traversal techniques Table 51 gives a more comprehensivecomparison1 between the boilerplate Uniplate and Syntactic Zipper approaches to treetraversal Code for the experiments is provided in Listing B1
Four operations are considered The consts functions are the same as those describedin Section 42 returning a list of all the constants in a logic tree The searchOr functionsreturn a list of all the OrZ terms in a tree The eval functions use tree transformationto reduce a logic tree containing only constants to a boolean value The subs functionsinverts the value of every constant in the tree
In every case the Syntactic Zipper takes the longest to perform both the query andmodification operations However its awareness of context allows immediate traversalsto sibling and parent nodes This type of operation in addition to being able to hold azipper at a particular location facilitates interactive targeted transformation
We therefore argue that where interactive transformation is a requirement despite theperformance overheads the Syntactic Zipper is a more acceptable solution than usingthe boilerplate or Uniplate techniques
52 Performance of the Supercompiler
The supercompiler is an optimising transformation To gauge its success we need todetermine the execution time of supercompiled programs relative to their original forms
However this is not the only measure of a compiler optimisation We investigate thetime that the supercompilation process takes and provide other statistics to give betterinsight into the supercompilation process
1Values were obtained using the UNIX time command on a 216 GHz Intel Core Duo Apple MacBook Prowith 2GB of RAM running Mac OS X 1056
63
5 Results and Evaluation
Table 51 Comparison of tree query and modification techniques (lower is better and worstscores are bold)
Performance for n nodes
Test Library 28 210 212 214 216
consts boilerplate 0004s 0005s 0008s 0018s 0053suniplate 0004s 0006s 0006s 0016s 0053szipper 0005s 0006s 0015s 0056s 0241s
searchOr boilerplate 0008s 0023s 0101s 0316s 1832suniplate 0007s 0020s 0072s 0319s 1821szipper 0048s 1046s 15949s gt 1m gt 1m
eval boilerplate 0004s 0005s 0005s 0004s 0006suniplate 0004s 0005s 0010s 0021s 0058szipper 0005s 0008s 0017s 0042s 0153s
subs boilerplate 0006s 0007s 0023s 0069s 0232suniplate 0005s 0007s 0021s 0061s 0219szipper 0006s 0012s 0029s 0128s 0514s
521 Test Programs
Five test programs are used to assess the performance of the Optimus Prime super-compiler They represent a variety of problem classes that should give an indication as tohow the supercompiler would operate on a wider set of programs
Perms The listing for the Perms test program can be found in Section C1 The programcalculates returns the size of the set of permutations for the list of the first four non-negative integers
Fibonacci The listing for the Fibonacci test program can be found in Section C2 Theprogram returns the first eighteen elements of the Fibonacci series The Fibonacci series isrepresented as an lazy infinite data structure and the integers themselves are representedby a data structure for Peano numerals
Twizzler The listing for the Twizzler test program can be found in Section C3 Thetwiz function takes the first integer of a list n and reverses the top n elements of thelist The twizzle function repeatedly applies twiz until the number 1 is at the top ofthe list The Twizzler problem asks from what starting states does the twizzle functionterminate with all the elements in order This program counts the number of orderedresults where the inputs are all the permutations 1 2 3 4
Word Count The listing for the Word Count test program can be found in Section C4The program finds the number of words separated by the space character in the stringquick brown fox jumps over the lazy dog Appendix D shows the supercompilationtrace for the word counting program and Listing E1 presents the final supercompiledcode Appendix D shows the effect of supercompilation on the word counting program
64
52 Performance of the Supercompiler
N-Queens The listing for the N-Queens test program can be found in Section C5 Theproblem asks in for an nxn size chessboard how many unique solutions are there toplacing n queens such that no queen is attacking another This program calculates theresult for the n = 10 instance
522 The Supercompilation Process
Each program is passed through the supercompiler Table 52 lists statistics concerningthe time the supercompilation process takes2 the number of lines of program codebefore and after supercompilation the number of definitions in the program before andafter supercompilation and how many residuals were produced prior to final inliningRatios are a f terbe f ore so that the supercompilation process producing less lines ofcode than the original results in a ratio that is less than one
Further statistics regarding the simplification process are listed in Table 53 Occur-rences of annotations relating to each simplification transformation are were countedusing the UNIX grep command
523 The Supercompiled Programs
Performance of the programs is measured in two ways Reductions are counted using theHUGS s +s statistics printing command Reduceron clock-ticks are simulated using acompiler and emulator provided by Matthew Naylor
For f-lite programs to be executable in HUGS several alterations need to be madeAny function that is defined in the program that also exists in the Haskell 98 pre-lude needs to be prevented from being imported This is done by adding a line likeimport Prelude hiding (mapconcatconcatMap) In addition any data constructor usedmust be defined explicitly For example data List a = Nil | Cons a (List a) is required forprograms that use list structures
Table 54 lists the execution performance of the test programs in both environmentsRatios are once again a f terbe f ore so that the supercompiled programs requiring lessReduceron ticks then the original program have ratios that are less than one
The N-Queens program appeared to be broken by the supercompilation An error wasintroduced by the supercompilation process such that the program no longer terminatesIn all other cases executed supercompiled code returned the same results as their originalforms
524 Discussion
The supercompilation process terminated for all five programs The time taken tosupercompile seems to be loosely related to the number of lines in the original sourcefiles but this is not a strict rule It is more closely aligned with the count of the numberof residuals produced as one would expect
Every supercompiled test program resulted in more lines of code than the originaldefinition This is probably due the ldquoduplicating code but not workrdquo (Mitchell 2008 chap4) behaviour of some simplification rules
2Supercompilation time is calculated using the UNIX time command on a 216 GHz Intel Core Duo AppleMacBook Pro with 2GB of RAM running Mac OS X 1056
65
5 Results and Evaluation
Table 52 Effects of supercompilation on code
Lines of Code Definitions
Program Time Before After Ratio Before After Ratio Residuals
Perms 6943s 47 4811 10236 12 16 133 27
Fibonacci 0597s 33 228 691 8 12 150 16
Twizzler 1134s 89 161 181 22 15 068 18
Word Count 0137s 39 83 213 8 5 063 11
N-Queens 26011s 77 400 520 21 16 076 58
Table 53 Supercompilation simplification statistics
Number of simplifications performedProgram appOfApp caseOfCons appToCase appToLet caseOfLet caseOfCase
Perms 3 74 0 0 49 59Fibonacci 0 20 0 0 5 15Twizzler 0 4 0 0 4 5Word Count 0 16 0 0 3 15N-Queens 3 21 0 0 17 18
Number of simplifications performedProgram substiuteVar letInCase inlineLet splitLet removeLet letInLet
Perms 0 0 264 12 247 0Fibonacci 0 0 84 0 84 0Twizzler 1 0 64 1 53 0Word Count 6 0 47 6 38 0N-Queens 1 0 203 3 187 0
Table 54 Comparison of the performance of supercompiled programs (lower is better)
Hugs Reductions Reduceron Clock-ticksProgram Before After Ratio Before After Ratio
Perms 594 455 077 2845 2208 078Fibonacci 236486 210307 089 655883 515720 079Twizzler 5732 5196 091 23244 21030 090Word Count 505 354 070 2107 1223 058N-Queens 15494927 NA NA 110636246 NA NA
66
53 Summary
The lsquopermsrsquo example produced a significant increase in the number of lines Thisappears to have been caused by the inlining of many application of a non-recursivefunction once supercompilation terminated It would be interesting to observe thepenalty caused if this inlining had not occurred
No discernible pattern can be seen in the relationship between the number of definitionsproduced and any other statistic The final number of definitions is determined by howmany recursive residuals are left by the final inlining process and is likely programspecific
It is notable that in none of the five test programs were the (a) application to caseexpression (b) application to a let expression (c) moving a let expression within a caseexpression (d) or nested let concatenation transformations ever used The first two areprobably due to partial evaluation not being necessary or properly exploited in theoriginal program code The latter two require further investigation to explain
The inlineLets transformation is heavily used due to the unfolding operation makinguse of lets to preserving sharing Similarly the removeLet transformation is heavily usedto remove excess let expressions left over from inlineLets
The caseOfCase and caseOfCons transformations are the driving force behind the su-percompilation strategy producing the evaluation effects that allow expressions to bereduced
The supercompiled forms of all except the N-Queens program indicated better per-formance than the original code in terms of Reduceron ticks and HUGS reductions Sometransformation appears to have changed the semantic meaning of the N-Queens programand this warrants further investigation
Performance gains appear to come from programs where the intermediate data struc-tures have been collapsed as would have been achieved by fusion and where separatefunctions have been composed The word counting program builds up a number ofintermediate data structures in its original form but the supercompiled version (List-ing E1) shows that these have been removed and the higher order functions have beenspecialised
53 Summary
The Syntactic Zipper library that lies at the heart of Optimus Prime does take longerto perform query and modification than some other generic tree traversal techniques Yetthe ability to traverse not just to a nodersquos children but to parents and siblings is essentialfunctionality for an interactive transformation tool
The Optimus Prime supercompiler has produced performance gains in all but onetest so far Further work is required to determine where semantic equivalence is lost in theN-Queens test Similarly our analysis of the results has shown that some simplificationtransformation are never triggered It is of interest to discover what constructs precipitatetheir use and whether they occur in general use
67
6 Conclusions and Further Work
This chapter first revisits the projectrsquos five core aims as laid out in Section 12 Withreference to these and other experience from the rest of this dissertation we shall proposetopics for further research
61 Satisfaction of Objectives
Interactive transformation The goal was ldquoto develop an interactive transformation tool OptimusPrime that can operate on the f-lite subset of Haskell 98rdquo An interactive tool has beendeveloped meeting all the requirements listed in Figure 315 The toolrsquos interactivetransformation interface is described in Section 45
FoldUnfold for f-lite In Section 34 we presented the Burstall and Darlington (1977)foldunfold transformation rules adapted for the particular constructs of f-liteSection 44 discussed how these rules have been implemented on the OptimusPrime platform
Supercompilation for f-lite Section 35 discussed the particulars of applying Turchin (1979)supercompilation to the f-lite subset of Haskell 98 A set of twelve transformationswere presented for the simplification of f-lite expressions The termination criteriaand residuation functions of (Mitchell 2008 chap 4) were changed to take intoaccount f-litersquos primitive functions
Optimus Prime supercompiler Using the blueprint described in Section 35 the supercom-pilation strategy was implemented in Optimus Prime for the f-lite language Itcan be executed using Optimus Primersquos interactive interface or autonomouslythrough a command-line tool
Effectiveness of the supercompiler In Chapter 5 we assessed the speed of the super-compilation process and its core components A selection of test programs weresupercompiled using the tool Four out of five of the test programs showed aperformance gain for the supercompiled version compared with the original formOne test program was rendered non-executable by the supercompilation processand possible reasons for its failure were discussed
Overall it would appear that the objectives outlined in Section 12 have been largelyachieved However further work is required to reduce the time that supercompilationtakes and to find the reason for its incompleteness
68
62 Taking Zippers out of Context
62 Taking Zippers out of Context
During traversal at every point in the tree the Syntactic Zipper (Section 41) maintainsa full record of its context That is a description of its position in the tree a reversedpointer to its parent and the scope stacks for each variable
For interactive transformation this information is essential for a user to move fromnode to related node without having to traverse down the complete tree Duringautomatic strategy-driven transformation these types of movements are not performedso the contextual data being carried is a hindrance to performance
The Syntactic Zipper representation could be replaced with a standard tree structurewhen performing autonomous transformations However transformations would needto be reimplemented for this new structure eliminating some of the benefits of usinginteractive transformation to develop the techniques
A comprise between the two positions would be to produce an alternative SyntacticZipper library which could be substituted in for the existing one during unaided trans-formation This new Nimble Syntactic Zipper library would only permit the decent throughthe zipper This restriction means that the contextual information is no longer requiredThe reversed pointer to the parent can be omitted and only the top element of the scopestacks need be retained
This would likely result in similar performance to that of the Uniplate library withouthaving to rewrite the transformations for a new structure and API
63 Optimisation of the Homeomorphic Embedding Relation
Another bottleneck in the supercompilation process is the use of the homeomorphicembedding relation as a termination criterion Each previous residual body is comparedto the expression currently being supercompiled The comparison traverses the fullexpression looking for sections that are similar the the derivation in question
The time it takes to find a homeomorphic embedding is bound in the worst case bythe current number of derivations available and the depth of the current instance Theformer will increase with respect to supercompilation time and the later fluctuates
Mitchell (2008 chap 2) suggests an the use of Sillmanrsquos (1989) algorithm to improveperformance through the use of a memoization table Mitchell reckons that there areperformance gains to made from using this technique and it would be interesting to seeif it would work in the context of our supercompiler variant for f-lite
64 Supercompilation of Primitive Functions
Our interpretation of the supercompilation strategy treats primitive function applicationsas constructs that cannot be simplified This is may be a premature assumption
An equality primitive (==) being applied to the same constructor integer or thesame strict variable could be evaluated to True Similarly (+) the addition of twointegers can be resolved to the result of the addition The emit primitive produces sideeffects by emitting results from the Reduceron but to the program it merely acts as the
69
6 Conclusions and Further Work
identity function At supercompilation time any input to the function could be passedoutward to a case expression for matching
There is plenty of scope for performing further supercompilation through primitivefunctions However termination criteria and generalisation functions will have to beadjusted for the loss structural information
It could also be argued that special rules for numeric primitives are necessary inthis context The Reduceron 2 is orientated towards symbolic computations those thatmanipulate structures rather than perform arithmetic As such programs written for thisplatform are likely to involve little arithmetic computation
However the primitive comparison of atomic values such as integers and constructorsoccurs in many programs The ideas of the Soslashrensen et al (2008) positive supercompiler orthe more general case passing on both positive and negative information could be usedto pass on inferred information about variable values
65 Closing Remarks
The use of an interactive program transformation tool to prototype transformationsallows both user and developer to step through strategies and directly observe the effecton code We found that this substantially aided the debugging of the Optimus Primesupercompiler
The internal representation of abstract syntax trees in Optimus Prime the SyntacticZipper is a very appropriate data structure for interactive transformation With furtherrefinement (as discussed in Section 62) it can be better utilised in autonomous unaidedtransformation systems
The results presented in Section 52 show that there are significant performance gainsto be made from the use of the supercompilation strategy on Reduceron 2 programs Thecurrent form of the Optimus Prime supercompiler is only reasonable to use on smallexperimental programs However the Reduceron 2 is similarly not yet ready for usewith large scale applications Both of these platforms can grow and stabilise in parallelto handle large applications with time
70
A FoldUnfold Transformation Transcripts
A1 Fibonacci Transformation Transcript
Based on the example in Burstall and Darlington (1977) Discussed in Section 34
1 minus Eureka minus2 f i b x = case ( lt=) x 1 of 3 True rarr 1 4 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )5 6 minus I n s t a n t i a t e lsquo f i b lsquo wi th lsquo x lt= 2 lsquo and lsquo x gt 2 lsquo minus 7 f i b x = case ( lt=) x 1 of 8 True rarr 1 9 Fa lse rarr case (==) x 2 of
10 True rarr ( + ) ( f i b ((minus ) 2 2 ) ) ( f i b ((minus ) 2 1 ) ) 11 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) ) ( f i b ((minus ) x 1 ) )12 13 14 minus P r i m i t i v e laws on lsquo(minus ) lsquo and u n f o l d lsquo f i b lsquo minus 15 f i b x = case ( lt=) x 1 of 16 True rarr 1 17 Fa lse rarr case (==) x 2 of 18 True rarr ( + ) ( f i b 0 ) ( f i b 1 ) 19 Fa lse rarr ( + ) ( f i b ((minus ) x 2 ) )20 ( ( + ) ( f i b ((minus ) x 3 ) ) ( f i b ((minus ) x 2 ) ) )21 22 23 minus Unfold lsquo f i b lsquo s p r i m i t i v e laws on lsquo ( + ) lsquo and a b s t r a c t lsquo f i b 2 lsquo minus 24 f i b x = case ( lt=) x 1 of 25 True rarr 1 26 Fa lse rarr case (==) x 2 of 27 True rarr 2 28 Fa lse rarr l e t f i b 2 = f i b ((minus ) x 2 ) in29 ( + ) ( f i b ((minus ) x 3 ) ) ( ( + ) f i b 2 f i b 2 )30 31
Listing A1 An example of optimising a function using foldunfold expressed in f-lite
71
A FoldUnfold Transformation Transcripts
A2 AppendElem Proof Transcript
Discussed in Section 34
1 minus P r e l i m i n a r y D e f i n i t i o n s minus2 elem x ys = case ys of 3 Nil rarr Fa lse 4 Cons y1 ys1 rarr or ( ( = = ) x y1 ) ( elem x ys1 )5 6 or x y = case x of 7 True rarr True 8 Fa lse rarr y 9
10 append xs ys = case xs of 11 Nil rarr ys 12 Cons x1 xs1 rarr Cons x1 ( append xs1 ys )13 14
15 minus H y p o t h e s i s minus 16 hLhs v xs ys = elem v ( append xs ys ) 17 hRhs v xs ys = or ( elem v xs ) ( elem v ys ) 18
19 minus I n s t a n t i a t e xs in h y p o t h e s i s wi th _|_ Ni l and ( Cons x xs ) minus 20 hLhs v xs ys = case xs of 21 _|_ rarr hLhs1 v vs 22 Nil rarr hLhs2 v vs 23 Cons x xs rarr hLhs3 v vs x xs 24 25 hRhs v xs ys = case xs of26 _|_ rarr hRhs1 v vs 27 Nil rarr hRhs2 v vs 28 Cons x xs rarr hRhs3 v vs x xs 29 30
31 minus _|_ c a s e minus32 hLhs1 v ys = elem v ( append _|_ ys ) 33 hRhs1 v ys = or ( elem v _|_ ) ( elem v ys ) 34 minus S t r i c t argument in lsquo append lsquo and lsquo elem lsquo minus 35 hLhs1 v ys = elem v _|_ 36 hRhs1 v ys = or _|_ ( elem v ys ) 37 minus S t r i c t argument in lsquo elem lsquo and lsquo or lsquo minus 38 hLhs1 v ys = _|_ 39 hRhs1 v ys = _|_ 40 minus QED minus 41 hLhs1 v ys = hRhs1 v ys 42
43 minus Nil c a s e minus44 hLhs2 v ys = elem v ( append Nil ys ) 45 hRhs2 v ys = or ( elem v Nil ) ( elem v ys ) 46 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 47 hLhs2 v ys = elem v ys 48 hRhs2 v ys = or Fa l se ( elem v ys ) 49 minus Unfold lsquo or lsquo minus
72
A2 AppendElem Proof Transcript
50 hRhs2 v ys = elem v ys 51 minus QED minus 52 hLhs2 v ys = hRhs2 v ys 53
54 minus ( Cons x xs ) c a s e minus55 hLhs3 v ys x xs = elem v ( append ( Cons x xs ) ys ) 56 hRhs3 v ys x xs = or ( elem v ( Cons x xs ) ) ( elem v ys ) 57 minus Unfold lsquo append lsquo and lsquo elem lsquo minus 58 hLhs3 v ys x xs = elem v ( Cons x ( append xs ys ) ) 59 hRhs3 v ys x xs = or ( or ( ( = = ) v x ) ( elem v xs ) ) ( elem v ys ) 60 minus Unfold lsquo elem lsquo and a p p l y law o f a s s o c i a t i v i t y on lsquo or lsquo minus 61 hLhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 62 hRhs3 v ys x xs = or ( ( = = ) v x ) ( or ( elem v xs ) ( elem v ys ) ) 63 minus I n d u c t i v e h y p o t h e s i s so f o l d u n f o l d hRhs hLhs minus 64 hRhs3 v ys x xs = or ( ( = = ) v x ) ( elem v ( append xs ys ) ) 65 minus QED minus 66 hLhs3 v ys x xs = lRhs v ys x xs
Listing A2 Example of a foldunfold proof held in f-lite syntax
73
B Tree Traversal Comparison
This code is used to compare the performance three tree traversal techniques in Section 51(B) Boilerplate (U) Uniplate and (Z) Syntactic Zipper
1 import Zipper2 import Data Generics Un ip la te S t r3 import Data Generics S t r4 import F r e s h S t a t e5 import Data Maybe6
7 minusminus D e f i n i t i o n o f t h e LogicZ d a t a t y p e8 data LogicZ = AndZ ( ZipParent LogicZ ) ( ZipParent LogicZ )9 | OrZ ( ZipParent LogicZ ) ( ZipParent LogicZ )
10 | NotZ ( ZipParent LogicZ )11 | VariableZ S t r i n g12 | ConstantZ Bool13
14 minusminus I n s t a n c e o f U n i p l a t e l i b r a r y f o r LogicZ15 instance Uniplate LogicZ where16 un ip la t e (AndZ (N x H) (N y H) ) = ( Two (One x ) (One y )17 (Two (One x rsquo ) (One y rsquo ) ) rarr
AndZ (N x rsquo H) (N y rsquo H) )18 un ip la t e (OrZ (N x H) (N y H) ) = ( Two (One x ) (One y )19 (Two (One x rsquo ) (One y rsquo ) ) rarr
OrZ (N x rsquo H) (N y rsquo H) )20 un ip la t e ( NotZ (N x H) ) = ( One x (One x rsquo ) rarr
NotZ (N x rsquo H) )21 un ip la t e x = ( Zero Zero rarr x )22
23 minusminus I n s t a n c e o f S y n t a c t i c Z ip pe r l i b r a r y f o r LogicZ24 instance Zippable LogicZ where25 hole i fy rsquo 0 h (AndZ x y ) = (AndZ h y x )26 hole i fy rsquo 1 h (AndZ x y ) = (AndZ x h y )27 hole i fy rsquo 0 h (OrZ x y ) = (OrZ h y x )28 hole i fy rsquo 1 h (OrZ x y ) = (OrZ x h y )29 hole i fy rsquo 0 h ( NotZ x ) = ( NotZ h x )30
31 expWidth rsquo ( VariableZ _ ) = 0
32 expWidth rsquo ( ConstantZ _ ) = 0
33 expWidth rsquo ( NotZ _ ) = 1
34 expWidth rsquo _ = 2
35
36 addBindings _ _ = id37 removeBindings _ = id38
39 i n i t F r e s h = undefined
74
40
41 minusminus Tree g e n e r a t i o n where n i s t r e e d e p t h Number o f nodes = 2 ^ n42 t e s t Z I n t rarr LogicZ43 t e s t Z 0 = ConstantZ True44 t e s t Z n | odd n = AndZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)45 | otherwise = OrZ (N ( t e s t Z ( n minus 1 ) ) H) (N ( t e s t Z ( n minus 1 ) )
H)46
47 minusminus Consts f u n c t i o n48 constsB LogicZ rarr [ Bool ]49 constsB (AndZ (N x H) (N y H) ) = constsB x ++ constsB y50 constsB (OrZ (N x H) (N y H) ) = constsB x ++ constsB y51 constsB ( NotZ (N x H) ) = constsB x52 constsB ( VariableZ _ ) = [ ]53 constsB ( ConstantZ x ) = [ x ]54
55 constsU LogicZ rarr [ Bool ]56 constsU l = [ x | ConstantZ x ltminus universe l ]57
58 constsZ Zip LogicZ rarr [ Bool ]59 constsZ l = [ x | ( _ N ( ConstantZ x ) _ _ ) ltminus decendants l ]60
61 minusminus S e a r c h f u n c t i o n62 searchOrsB LogicZ rarr [ LogicZ ]63 searchOrsB (AndZ (N x H) (N y H) ) = searchOrsB x ++ searchOrsB y64 searchOrsB l (OrZ (N x H) (N y H) ) = ( l searchOrsB x ) ++ searchOrsB y65 searchOrsB ( NotZ (N x H) ) = searchOrsB x66 searchOrsB ( VariableZ _ ) = [ ]67 searchOrsB ( ConstantZ x ) = [ ]68
69 searchOrsU LogicZ rarr [ LogicZ ]70 searchOrsU l = [ l rsquo | l rsquo(OrZ _ _ ) ltminus universe l ]71
72 searchOrsZ Zip LogicZ rarr [ Zip LogicZ ]73 searchOrsZ l = [ l rsquo | l rsquo( _ N (OrZ _ _ ) _ _ ) ltminus decendants l ]74
75 minusminus Eval f u n c t i o n76 evalB LogicZ rarr Bool77 evalB (AndZ (N x H) (N y H) ) = evalB x ampamp evalB y78 evalB (OrZ (N x H) (N y H) ) = evalB x || evalB y79 evalB ( NotZ (N x H) ) = not ( evalB x )80 evalB ( VariableZ _ ) = e r r o r Can rsquo t evaluate v a r i a b l e 81 evalB ( ConstantZ x ) = x82
83 evalU LogicZ rarr Bool84 evalU l = case rewr i te eval l of ConstantZ x rarr x 85 where86 eval (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x ampamp y )87 eval (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) = J u s t $
ConstantZ ( x || y )
75
B Tree Traversal Comparison
88 eval ( NotZ (N ( ConstantZ x ) H) ) = J u s t $ ConstantZ ( not x )89 eval _ = Nothing90
91 evalZ Zip LogicZ rarr Bool92 evalZ l = case runDead ( normalise eval l ) of ( _ N ( ConstantZ x ) _ _
) rarr x 93 where94 eval ( _ N (AndZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x ampamp y )95 eval ( _ N (OrZ (N ( ConstantZ x ) H) (N ( ConstantZ y ) H) ) _ _ )
= return $ ConstantZ ( x || y )96 eval ( _ N ( NotZ (N ( ConstantZ x ) H) ) _ _ ) = return $
ConstantZ ( not x )97 eval _ = die98
99 minusminus Subs f u n c t i o n100 subsB LogicZ rarr LogicZ101 subsB (AndZ (N x H) (N y H) ) = (AndZ (N ( subsB x ) H) (N ( subsB y ) H)
)102 subsB (OrZ (N x H) (N y H) ) = (OrZ (N ( subsB x ) H) (N ( subsB y ) H) )103 subsB ( NotZ (N x H) ) = ( NotZ (N ( subsB x ) H) )104 subsB ( ConstantZ x ) = ConstantZ ( not x )105 subsB x = x106
107 subsU LogicZ rarr LogicZ108 subsU = descend subs109 where110 subs ( ConstantZ x ) = ConstantZ ( not x )111 subs x = x112
113 subsZ Zip LogicZ rarr Zip LogicZ114 subsZ z = runDead ( updateSubtree subs z )115 where116 subs ( _ N ( ConstantZ x ) _ _ ) = return ( ConstantZ ( not x ) )117 subs x = die
Listing B1 Code for tree traversal tests
76
C Test Programs
The programs described in these listings are explained in Subsection 521
C1 Perms mdash Permiatation Counting
1 2 main = count ( take 4 i n f ) 3
4 count xs = length ( perms xs ) 5
6 append xs ys = case xs of 7 Nil rarr ys 8 Cons z zs rarr Cons z ( append zs ys )9
10
11 concat xs = case xs of 12 Nil rarr Nil 13 Cons y ys rarr append y ( concat ys )14 15
16 concatMap f xs = concat (map f xs ) 17
18 length xs = case xs of 19 Nil rarr 0 20 Cons y ys rarr ( + ) 1 ( length ys )21 22
23 i n s e r t i o n s x ys = case ys of 24 Nil rarr Cons ( Cons x Nil ) Nil 25 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )26 27
28 perms xs = case xs of 29 Nil rarr Cons Nil Nil 30 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )31 32
33 map f xs = case xs of 34 Nil rarr Nil 35 Cons y ys rarr Cons ( f y ) (map f ys )36 37
38 inc x = ( + ) 1 x
77
C Test Programs
39
40 i n f = Cons 1 (map inc i n f ) 41
42 take n xs = case (==) n 0 of 43 True rarr Nil 44 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 45 46
Listing C1 Code for the lsquopermsrsquo test program
78
C2 Fibonacci mdash Calculation of Fibonacci Numbers
C2 Fibonacci mdash Calculation of Fibonacci Numbers
1 2 values = I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I ( I Z) ) ) )
) ) ) ) ) ) ) ) ) ) ) ) ) 3
4 main = takeF values f i b s 5
6 f i b s = mapF f i b i n f F 7
8 mapF f ds = case ds of 9 Nil rarr Nil
10 Cons e es rarr Cons ( f e ) (mapF f es )11 12
13 i n f F = Cons Z (mapF I i n f F ) 14
15 takeF gs hs = case gs of 16 Z rarr Nil 17 I i rarr case hs of 18 Cons j j s rarr Cons j ( takeF i j s ) 19 20 21
22 plus a c = case a of 23 Z rarr c 24 I b rarr I ( plus b c ) 25 26
27 f i b x = case x of 28 Z rarr I Z 29 I y rarr case y of 30 Z rarr I Z 31 I z rarr plus ( f i b z ) ( f i b ( I z ) ) 32 33 34
Listing C2 Code for the lsquofibonaccirsquo test program
79
C Test Programs
C3 Twizzler mdash The Twizzler Problem
1 2 main = count 4 3
4 map f xs = case xs of 5 Nil rarr Nil 6 Cons y ys rarr Cons ( f y ) (map f ys )7 8
9 append xs ys = case xs of 10 Nil rarr ys 11 Cons z zs rarr Cons z ( append zs ys )12 13
14 concat xs = case xs of 15 Nil rarr Nil 16 Cons y ys rarr append y ( concat ys )17 18
19 concatMap f xs = concat (map f xs ) 20
21 length xs = case xs of 22 Nil rarr 0 23 Cons y ys rarr ( + ) 1 ( length ys )24 25
26 i n s e r t i o n s x ys = case ys of 27 Nil rarr Cons ( Cons x Nil ) Nil 28 Cons z zs rarr Cons ( Cons x ( Cons z zs ) ) (map ( Cons z ) ( i n s e r t i o n s
x zs ) )29 30
31 perms xs = case xs of 32 Nil rarr Cons Nil Nil 33 Cons y ys rarr concatMap ( i n s e r t i o n s y ) ( perms ys )34 35
36 take n xs = case (==) n 0 of 37 True rarr Nil 38 Fa lse rarr case xs of Cons y ys rarr Cons y ( take ((minus ) n 1 ) ys ) 39 40
41 drop n xs = case (==) n 0 of 42 True rarr xs 43 Fa lse rarr case xs of Cons y ys rarr drop ((minus ) n 1 ) ys 44 45
46 head xs = case xs of Cons y ys rarr y 47
48 reverse xs = r e v e r s e I n t o xs Nil 49
80
C3 Twizzler mdash The Twizzler Problem
50 r e v e r s e I n t o xs ys = case xs of 51 Nil rarr ys 52 Cons z zs rarr r e v e r s e I n t o zs ( Cons z ys )53 54
55 twiz xs = l e t n = head xs in append ( reverse ( take n xs ) ) ( dropn xs )
56
57 twizz le xs = case (==) ( head xs ) 1 of 58 True rarr xs 59 Fa lse rarr twizz le ( twiz xs )60 61
62 f i l t e r f xs = case xs of 63 Nil rarr Nil 64 Cons y ys rarr case f y of 65 True rarr Cons y ( f i l t e r f ys ) 66 Fa lse rarr f i l t e r f ys 67 68 69
70 unsorted xs = case xs of 71 Nil rarr Fa lse 72 Cons y ys rarr case ys of 73 Nil rarr Fa lse 74 Cons z zs rarr case ( lt=) y z of 75 True rarr unsorted ys 76 Fa lse rarr True77 78 79 80
81 inc x = ( + ) 1 x 82
83 i n f = Cons 1 (map inc i n f ) 84
85 input n = ( ( take n i n f ) ) 86
87 operat ion xs = (map twizz le ( perms xs ) ) 88
89 count n = length ( f i l t e r unsorted ( operat ion ( input n ) ) ) 90
Listing C3 Code for the lsquotwizzlerrsquo test program
81
C Test Programs
C4 Word Count mdash Counting Words in a String
1 2 values = quick brown fox jumps over the lazy dog 3
4 main = wc values 5
6 wc s = length ( words s ) 7
8 length xs = case xs of 9 Nil rarr 0
10 Cons y ys rarr ( + ) 1 ( length ys )11 12
13 i sSpace x = (==) rsquo rsquo x 14
15 words s = l e t x = dropWhile isSpace s in case x of 16 Nil rarr Nil 17 Cons t t s rarr case break isSpace x of 18 Pair w y rarr Cons w ( words y )19 20 21
22 dropWhile f xs = case xs of 23 Nil rarr Nil 24 Cons y ys rarr case f y of 25 True rarr dropWhile f ys 26 Fa lse rarr Cons y ys27 28 29
30 break f xs = case xs of 31 Nil rarr Pair Nil Nil 32 Cons y ys rarr case f y of 33 True rarr Pair Nil xs 34 Fa lse rarr case break f ys of 35 Pair l s r s rarr Pair ( Cons y l s ) r s36 37 38 39
Listing C4 Code for the lsquowcrsquo test program
82
C5 N-Queens mdash N = 10 Instance
C5 N-Queens mdash N = 10 Instance
1 2 t a i l i n s = case i n s of Cons x xs rarr xs 3
4 one p in s = case i n s of 5 Nil rarr Nil 6 Cons x xs rarr case p x of True rarr Cons x Nil Fa l se rarr one p xs
7 8
9 map f i ns = case i n s of 10 Nil rarr Nil 11 Cons x xs rarr Cons ( f x ) (map f xs )12 13
14 append in s ys = case i ns of 15 Nil rarr ys 16 Cons x xs rarr Cons x ( append xs ys )17 18
19 concatMap f i ns = case i n s of 20 Nil rarr Nil 21 Cons x xs rarr append ( f x ) ( concatMap f xs ) 22 23
24 length i ns = case i ns of 25 Nil rarr 0 26 Cons x xs rarr ( + ) 1 ( length xs ) 27 28
29 r e p l i c a t e n x = case (==) n 0 of 30 True rarr Nil 31 Fa lse rarr Cons x ( r e p l i c a t e ((minus ) n 1 ) x ) 32 33
34 l = 0 35 r = 1 36 d = 2 37
38 eq x y = (==) x y 39
40 l e f t xs = map ( one ( eq l ) ) ( t a i l xs ) 41 r i g h t xs = Cons Nil (map ( one ( eq r ) ) xs ) 42 down xs = map ( one ( eq d ) ) xs 43
44 merge xs ys = case xs of 45 Nil rarr Nil 46 Cons x1 xs1 rarr case ys of 47 Nil rarr xs 48 Cons y1 ys1 rarr Cons ( append x1 y1 ) ( merge xs1 ys1 )49
83
C Test Programs
50 51
52 next mask = merge ( merge (down mask ) ( l e f t mask ) ) ( r i g h t mask ) 53
54 f i l l i n s = case i n s of 55 Nil rarr Nil 56 Cons x xs rarr append ( l r d x xs ) (map ( Cons x ) ( f i l l xs ) ) 57 58
59 l rd i n s ys = case i n s of 60 Nil rarr Cons ( Cons ( Cons l ( Cons r ( Cons d Nil ) ) ) ys ) Nil 61 Cons x xs rarr Nil 62 63
64 solve n mask =65 case (==) n 0 of 66 True rarr Cons Nil Nil 67 Fa lse rarr concatMap ( s o l ((minus ) n 1 ) ) ( f i l l mask ) 68 69
70 s o l n row = map ( Cons row ) ( solve n ( next row ) ) 71
72 nqueens n = length ( solve n ( r e p l i c a t e n Nil ) ) 73
74 main = emit In t ( nqueens 10 ) 0 75
Listing C5 Code for the lsquonqueensrsquo test program
84
D Supercompilation Report for Word Count
This is the literate output of the Optimus Prime supercompiler for the Word Counttest program
85
D Supercompilation Report for Word Count
Reachable functions
main
main = wc values
wc
wc s = length (words s)equiv Inline lsquolengthrsquo Inline Lets Remove Lets
wc s = case words s ofNil rarr 0Cons y1 ys1 rarr (+) 1 (length ys1 )
equiv Inline lsquowordsrsquo Substitute Variables Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Case of ConstructorRemove Lets Case of Case Case of Constructor Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc0rsquo [20102]Generalised
wc s = case dropWhile isSpace s ofNil rarr 0Cons t1 ts1 rarr case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc1rdquordquowc2rdquo]
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr wc2 t1 ts1
equiv Final inlining Inline Lets Case of a Let Inline Lets Remove Lets Inline Lets Case of Case Inline Lets Remove LetsCase of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline LetsRemove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of ConstructorInline Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc s = case wc1 s ofNil rarr 0Cons t1 ts1 rarr case (equiv) 32 t1 of
True rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y23 ys18 rarr case (equiv) 32 y23 of
True rarr (+) 1 (wc (Cons y23 ys18 ))False rarr case ys18 of
Nil rarr (+) 1 (wc Nil)Cons y24 ys19 rarr case (equiv) 32
y24 ofTrue rarr (+) 1
(wc(Consy24ys19 ))
False rarr case ys19 ofNil rarr (+)
1(wcNil)
Consy25ys20 rarr case (equiv)
32y25 ofTrue rarr (+)
1(wc(Consy25ys20 ))
False rarr case wc9ys20 ofPair
ls10rs12 rarr (+)
1
1
86
(wcrs12 )
values
values = Cons 113 (Cons 117 (Cons 105 (Cons 99 (Cons 107 (Cons 32 (Cons 98(Cons 114 (Cons 111 (Cons 119 (Cons 110 (Cons 32 (Cons 102 (Cons 111 (Cons120 (Cons 32 (Cons 106 (Cons 117 (Cons 109 (Cons 112 (Cons 115 (Cons 32(Cons 111 (Cons 118 (Cons 101 (Cons 114 (Cons 32 (Cons 108 (Cons 97 (Cons122 (Cons 121 (Cons 32 (Cons 100 (Cons 111 (Cons 103Nil))))))))))))))))))))))))))))))))))
wc1
bull Residual of wcwc1 s = dropWhile isSpace s
equiv Inline lsquodropWhilersquo Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc10rsquo [0102] Generalised wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquowc3rdquo] wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr wc3 y3 ys2
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Remove Lets wc1 s = case s of
Nil rarr NilCons y3 ys2 rarr case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc9
bull Residual of wc8wc9 ys4 = break isSpace ys4
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc90rsquo [00202]Generalised
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case isSpace y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case break isSpace ys5 of
Pair ls3 rs5 rarr Pair (Cons y6 ls3 ) rs5equiv Residuated [rdquowc4rdquo]
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr wc4 y6 ys5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets Inline Lets InlineLets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets Inline Lets Remove Lets
wc9 ys4 = case ys4 ofNil rarr Pair Nil NilCons y6 ys5 rarr case (equiv) 32 y6 of
True rarr Pair Nil (Cons y6 ys5 )False rarr case ys5 of
Nil rarr Pair (Cons y6 Nil) NilCons y17 ys12 rarr case (equiv) 32 y17 of
True rarr Pair (Cons y6 Nil) (Cons y17ys12 )
False rarr case wc9 ys12 ofPair ls5 rs7 rarr Pair (Cons y6
(Cons y17ls5 )) rs7
2
87
D Supercompilation Report for Word Count
Obsolete functions
wc3
bull Residual of wc1 Homeomorphically embeds lsquowc10rsquo [01] Generalisedwc3 y3 ys2 = case isSpace y3 of
True rarr dropWhile isSpace ys2False rarr Cons y3 ys2
equiv Residuated [rdquoisSpacerdquordquowc1rdquo] wc3 y3 ys2 = case isSpace y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
equiv Final inlining Inline Lets Remove Lets wc3 y3 ys2 = case (equiv) 32 y3 of
True rarr wc1 ys2False rarr Cons y3 ys2
wc8
bull Residual of wc7 Homeomorphically embeds lsquowc60rsquo []wc8 ys4 t1 y5 = case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc9rdquo]
wc8 ys4 t1 y5 = case wc9 ys4 ofPair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2
wc7
bull Residual of wc6 Homeomorphically embeds lsquowc60rsquo [02]wc7 y5 t1 ys4 = case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquoisSpacerdquordquowc8rdquo]
wc7 y5 t1 ys4 = case isSpace y5 ofTrue rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr wc8 ys4 t1 y5
equiv Final inlining Inline Lets Remove Lets Inline Lets Inline Lets Inline Lets Remove Lets wc7 y5 t1 ys4 = case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls6 rs8 rarr Pair (Cons t1 (Cons y5 ls6 )) rs8
wc6
bull Residual of wc4wc6 ts1 t1 = case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Case of a Let Inline Lets Remove Lets Case of Case Case of Constructor
Inline Lets Case of Case Inline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case ofConstructor Inline Lets Inline Lets Remove Lets Inline Lets Remove Lets Homeomorphically embeds lsquowc60rsquo [0202]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case isSpace y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case break isSpace ys4 of
Pair ls2 rs2 rarr Pair (Cons t1 (Cons y5 ls2 )) rs2equiv Residuated [rdquowc7rdquo]
wc6 ts1 t1 = case ts1 ofNil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr wc7 y5 t1 ys4
equiv Final inlining Inline Lets Inline Lets Inline Lets Remove Lets wc6 ts1 t1 = case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y5 ys4 rarr case (equiv) 32 y5 of
True rarr Pair (Cons t1 Nil) (Cons y5 ys4 )False rarr case wc9 ys4 of
Pair ls7 rs9 rarr Pair (Cons t1 (Cons y5 ls7 )) rs9
3
88
wc4
bull Residual of wc2wc4 t1 ts1 = break isSpace (Cons t1 ts1 )
equiv Inline lsquobreakrsquo Substitute Variables Inline Lets Split Lets Case of Constructor Inline Lets Inline Lets Remove LetsInline Lets Inline Lets Remove Lets
wc4 t1 ts1 = case isSpace t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Inline lsquoisSpacersquo Inline Lets Remove Lets Homeomorphically embeds lsquoisSpace1rsquo [0] Generalised
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr case break isSpace ts1 of
Pair ls1 rs1 rarr Pair (Cons t1 ls1 ) rs1equiv Residuated [rdquowc6rdquo]
wc4 t1 ts1 = case (equiv) 32 t1 ofTrue rarr Pair Nil (Cons t1 ts1 )False rarr wc6 ts1 t1
equiv Final inlining Inline Lets Inline Lets Remove Lets wc4 t1 ts1 = case (equiv) 32 t1 of
True rarr Pair Nil (Cons t1 ts1 )False rarr case ts1 of
Nil rarr Pair (Cons t1 Nil) NilCons y22 ys17 rarr case (equiv) 32 y22 of
True rarr Pair (Cons t1 Nil) (Cons y22 ys17 )False rarr case wc9 ys17 of
Pair ls8 rs10 rarr Pair (Cons t1 (Cons y22ls8 )) rs10
wc5
bull Residual of wc2 Homeomorphically embeds lsquowc0rsquo [2] Generalisedwc5 y2 = (+) 1 (length (words y2 ))
equiv Residuated [rdquowcrdquo] wc5 y2 = (+) 1 (wc y2 )
wc2
bull Residual of wc Homeomorphically embeds lsquowc0rsquo [201] Generalisedwc2 t1 ts1 = case break isSpace (Cons t1 ts1 ) of
Pair w1 y2 rarr (+) 1 (length (words y2 ))equiv Residuated [rdquowc4rdquordquowc5rdquo]
wc2 t1 ts1 = case wc4 t1 ts1 ofPair w1 y2 rarr wc5 y2
equiv Final inlining Inline Lets Remove Lets Inline Lets Remove Lets Case of Case Inline Lets Inline Lets Inline Lets InlineLets Inline Lets Remove Lets Case of a Let Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Inline LetsInline Lets Remove Lets Case of Case Case of Constructor Inline Lets Split Lets Inline Lets Remove Lets Inline Lets RemoveLets Inline Lets Remove Lets Case of Case Inline Lets Case of Constructor Inline Lets Case of Case Inline Lets Remove LetsInline Lets Remove Lets Case of Constructor Inline Lets Split Lets Case of Case Inline Lets Case of Constructor Inline LetsCase of Case Inline Lets Remove Lets Inline Lets Remove Lets Case of Constructor Inline Lets Inline Lets Remove Lets
wc2 t1 ts1 = case (equiv) 32 t1 ofTrue rarr (+) 1 (wc (Cons t1 ts1 ))False rarr case ts1 of
Nil rarr (+) 1 (wc Nil)Cons y8 ys7 rarr case (equiv) 32 y8 of
True rarr (+) 1 (wc (Cons y8 ys7 ))False rarr case ys7 of
Nil rarr (+) 1 (wc Nil)Cons y11 ys10 rarr case wc4 y11 ys10 of
Pair ls4 rs6 rarr (+) 1 (wcrs6 )
length
length xs = case xs ofNil rarr 0Cons y ys rarr (+) 1 (length ys)
4
89
E Supercompiled Word Count
This is the tidy output of the Optimus Prime supercompiler for the Word Count testprogram
1 2 wc1 s = case s of 3 Nil rarr Nil 4 Cons y3 ys2 rarr case (==) 32
5 y3 of 6 True rarr wc1 ys2 7 Fa lse rarr Cons y3 ys2
8 9
10
11 wc9 ys4 = case ys4 of 12 Nil rarr Pair Nil Nil 13 Cons y6 ys5 rarr case (==) 32
14 y6 of 15 True rarr Pair Nil ( Cons y6 ys5 ) 16 Fa lse rarr case ys5 of 17 Nil rarr Pair ( Cons y6 Nil ) Nil 18 Cons y17 ys12 rarr case (==) 32
19 y17 of 20 True rarr Pair ( Cons y6 Nil ) ( Cons21 y17 ys12 ) 22 Fa lse rarr case wc9 ys12 of 23 Pair l s 5 rs7 rarr Pair ( Cons y6
24 ( Cons y17 l s 5 ) ) r s7
25 26 27 28 29 30
31 wc s = case wc1 s of 32 Nil rarr 0 33 Cons t1 t s 1 rarr case (==) 32
34 t1 of 35 True rarr ( + ) 1 (wc ( Cons t1
36 t s 1 ) ) 37 Fa lse rarr case t s 1 of 38 Nil rarr ( + ) 1 (wc Nil ) 39 Cons y23 ys18 rarr case (==) 32
40 y23 of 41 True rarr ( + ) 1 (wc ( Cons y23
42 ys18 ) )
90
43 Fa lse rarr case ys18 of 44 Nil rarr ( + ) 1 (wc Nil ) 45 Cons y24 ys19 rarr case (==) 32
46 y24 of 47 True rarr ( + ) 1 (wc ( Cons y24
48 ys19 ) ) 49 Fa lse rarr case ys19 of 50 Nil rarr ( + ) 1 (wc Nil ) 51 Cons y25 ys20 rarr case (==) 32
52 y25 of 53 True rarr ( + ) 1 (wc ( Cons y25
54 ys20 ) ) 55 Fa lse rarr case wc9 ys20 of 56 Pair l s 1 0 rs12 rarr ( + ) 1 (wc57 rs12 )58 59 60 61 62 63 64 65 66 67
68 values = Cons 113 ( Cons 117
69 ( Cons 105 ( Cons 99 ( Cons 107
70 ( Cons 32 ( Cons 98 ( Cons 114
71 ( Cons 111 ( Cons 119 ( Cons 110
72 ( Cons 32 ( Cons 102 ( Cons 111
73 ( Cons 120 ( Cons 32 ( Cons 106
74 ( Cons 117 ( Cons 109 ( Cons 112
75 ( Cons 115 ( Cons 32 ( Cons 111
76 ( Cons 118 ( Cons 101 ( Cons 114
77 ( Cons 32 ( Cons 108 ( Cons 97
78 ( Cons 122 ( Cons 121 ( Cons 32
79 ( Cons 100 ( Cons 111 ( Cons 103
80 Nil ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 81
82 main = wc values83
Listing E1 Supercompiled code for the lsquoword countrsquo test program
91
Bibliography
TOR Allwood and S Eisenbach Clase cursor library for a structured editor InProceedings of the ACM SIGPLAN symposium on Haskell pages 123ndash124 ACM 2008
Ronald E Anderson ACM Code of Ethics and Professional Conduct Commun ACM 35
(5)94ndash99 1992 ISSN 0001-0782 doi httpdoiacmorg101145129875129885
J Armstrong A history of Erlang In Proceedings of the third ACM SIGPLAN conference onHistory of programming languages pages 6ndash1 ACM New York NY USA 2007
FL Bauer Program development by stepwise transformations mdash The Project CIP ProgramConstruction 69 1979
ID Baxter C Pidgeon and M Mehlich DMSreg Program Transformations for Prac-tical Scalable Software Evolution In Proceedings of the 26th International Conference onSoftware Engineering pages 625ndash634 IEEE Computer Society Washington DC USA2004
R M Burstall and John Darlington A transformation system for developing recursiveprograms J ACM 2444ndash67 1977
WN Chin Safe fusion of functional expressions ACM SIGPLAN Lisp Pointers 5(1)11ndash20 1992
A Church An unsolvable problem of elementary number theory American Journal ofMathematics 58(2)345ndash363 1936
JR Cordy The TXL source transformation language Science of Computer Programming61(3)190ndash210 2006
JR Cordy CD Halpern and E Promislow TXL A rapid prototyping system for pro-gramming language dialects In Computer Languages 1988 Proceedings InternationalConference on pages 280ndash285 1988
Martin S Feather A system for assisting program transformation ACM Trans ProgramLang Syst 41ndash20 1982
Raphael Finkel Advanced Programming Language Design Addison Wesley 1st editionDecember 1995
Michael Anthony Firth A foldunfold transformation system for a non-strict language PhDthesis University of York 1990 222p
Free Software Foundation The GNU Compiler Collection 2009 URL httpgccgnuorg
Haskellorg GHC Optimisations 2008 URL httphaskellorghaskellwikiGHC_optimisations
92
Bibliography
P Hudak J Peterson and JH Fasel A gentle introduction to Haskell 98 1999 URLhttpwwwhaskellorgtutorial
Paul Hudak Conception evolution and application of functional programming lan-guages ACM Comput Surv 21359ndash411 1989
G Huet Functional pearl The zipper Journal of Functional Programming 7549ndash554 1997
PA Jonsson and J Nordlander Positive Supercompilation for a higher order call-by-value language In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposiumon Principles of programming languages pages 277ndash288 ACM 2009
Ralf Laumlmmel and Simon Peyton Jones Scrap your boilerplate a practical design patternfor generic programming ACM SIGPLAN Notices 38(3)26ndash37 March 2003 Proceed-ings of the ACM SIGPLAN Workshop on Types in Language Design and Implementa-tion (TLDI 2003)
R Milner M Tofte and R Harper The definition of Standard ML MIT press 1990
N Mitchell and C Runciman Uniform boilerplate and list processing In Proceedings ofthe ACM SIGPLAN workshop on Haskell workshop pages 49ndash60 ACM 2007
Neil Mitchell First order Haskell Presentation from BCTCS 2007 April 2007 URLhttpcommunityhaskellorg~ndmdownloadsslides-first_order_haskell-06_apr_2007pdf
Neil Mitchell Transformation and Analysis of Functional Programs PhD thesis Universityof York June 2008 URL httpcommunityhaskellorg~ndmdownloadspaper-transformation_and_analysis_of_functional_programs-4_jun_2008pdf
Neil Mitchell Losing functions without gaining data March 2009 URL httpcommunityhaskellorg~ndmdownloadsdraft-losing_functions_without_gaining_data-02_mar_2009pdf
Matthew Naylor and Colin Runciman The Reduceron Widening the von Neumannbottleneck for graph reduction using an FPGA In Implementation and Application ofFunctional Languages (IFL 2007 Revised Selected Papers) pages 129ndash146 Springer LNCS5083 2008
J Newburn All about monads v110 URL httphaskellorgall_about_monadshtmlindexhtml
B OrsquoSullivan DB Stewart and J Goerzen Real World Haskell OrsquoReilly Media Inc 2008
W Partain The NoFib benchmark suite of Haskell programs In Proceedings of the 1992Glasgow Workshop on Functional Programming pages 195ndash202 Springer-Verlag LondonUK 1992
JC Reynolds Definitional interpreters for higher-order programming languages In ACMAnnual ConferenceAnnual Meeting Proceedings of the ACM annual conference- Volume2 Boston Massachusetts United States pages 717ndash740 Association for ComputingMachinery Inc One Astor Plaza 1515 Broadway New York NY 10036-5701 USAbdquo1972
93
Bibliography
T Sheard and SP Jones Template meta-programming for Haskell ACM SIGPLANNotices 37(12)60ndash75 2002
Simon Peyton Jones et al The Haskell 98 language and libraries The revised reportJournal of Functional Programming 13(1)0ndash255 Jan 2003
DR Smith KIDS A knowledge-based software development system Automating SoftwareDesign pages 483ndash514 1991
MH Soslashrensen R Gluumlck and ND Jones A positive supercompiler Journal of FunctionalProgramming 6(06)811ndash838 2008
G Steele Common LISP the language Digital press 1990
Jonathan Stillman Computational problems in equational theorem proving PhD thesisAlbany NY USA 1989
D Syme The F programming language URL httpresearchmicrosoftcomprojectsfsharp
MA Tullsen PATH A Program Transformation System for Haskell PhD thesis YaleUniversity 2002
VF Turchin A supercompiler system based on the language Refal ACM SIGPLANNotices 14(2)46ndash54 1979
D Turner An overview of Miranda Research Topics in Functional Programming page 11990
MGJ van den Brand A van Deursen J Heering HA De Jong M de Jonge T KuipersP Klint L Moonen PA Olivier J Scheerder et al The ASF+ SDF Meta-Environmenta component-based language development environment Electronic Notes in TheoreticalComputer Science 44(2)3ndash8 2001
M van Eekelen and R Plasmeijer Concurrent Clean Language Report (version 20)University of Nijmegen 2001 URL httpcleancsrunldownloadClean20docCleanLangRep21pdf
Eelco Visser Program transformation with StrategoXT Rules strategies tools andsystems in StrategoXT-09 In C Lengauer et al editors Domain-Specific ProgramGeneration volume 3016 of Lecture Notes in Computer Science pages 216ndash238 Spinger-Verlag June 2004 URL httpwwwcsuunlresearchtechrepsUU-CS-2004-011html
Eelco Visser Tom Mens and Malcolm Wallace Program transformation 2004 URLhttpwwwprogram-transformationorgviewTransformProgramTransformationrev=123
P Wadler Deforestation Transforming programs to eliminate trees Theoretical ComputerScience 73(2)231ndash248 1990
P Wadler A prettier printer Journal of Functional Programming pages 223ndash244 1998
M Ward and H Zedan MetaWSL and meta-transformations in the fermaT transformationsystem In Conference on Computer Software and Applications Conference COMPSAC 200529th Annual International 2005
94
- Introduction
-
- Motivation
- Aims of this Project
- Structure of this Report
- Statement of Ethics
-
- Background and Review
-
- Functional Languages and Haskell
- Program Transformation
-
- The FoldUnfold Laws
- Defunctionalisation
- Deforestation and Fusion
- Supercompilation
-
- Program Transformation Tools
- The Reduceron and F-Lite
- Summary
-
- Problem Analysis
-
- Specification of Transformations
- The Core Language
-
- Preliminary Definitions
-
- Transformation Primitives for f-lite
-
- Inline Unfold
- Fold
- Residuate
-
- The FoldUnfold Laws
- Supercompilation Strategy
-
- Termination Criteria
- Unfolding Selection
- Simplification Transformations for f-lite
- Residuation Folding and Generalisation
- Inlining Unnecessary Residuals
-
- Requirements A Transformation Framework
- Summary
-
- Design and Implementation
-
- Representation through Syntactic Zippers
- Zipper Query and Modification
- Transformation Specification
- Transformation Specifics
-
- Low-Level Definitions
- Transformation Rules and Strategies
-
- Output and Control
-
- Pretty printing
- Interactive Control
- Autonomous Control
-
- Summary
-
- Results and Evaluation
-
- Performance of the Syntactic Zipper Library
- Performance of the Supercompiler
-
- Test Programs
- The Supercompilation Process
- The Supercompiled Programs
- Discussion
-
- Summary
-
- Conclusions and Further Work
-
- Satisfaction of Objectives
- Taking Zippers out of Context
- Optimisation of the Homeomorphic Embedding Relation
- Supercompilation of Primitive Functions
- Closing Remarks
-
- FoldUnfold Transformation Transcripts
-
- Fibonacci Transformation Transcript
- AppendElem Proof Transcript
-
- Tree Traversal Comparison
- Test Programs
-
- Perms mdash Permiatation Counting
- Fibonacci mdash Calculation of Fibonacci Numbers
- Twizzler mdash The Twizzler Problem
- Word Count mdash Counting Words in a String
- N-Queens mdash N = 10 Instance
-
- Supercompilation Report for Word Count
- Supercompiled Word Count
- Bibliography
-