tree oriented programming jeroen fokker. tree oriented programming nmany problems are like: input...

Tree Oriented Programming

Jeroen Fokker

Tree oriented programming

Many problems are like:

Inputtext

Outputtexttransform unparseprocess

Many problems are like:

Inputtext

Outputtexttransform unparseparse prettyprint

internal tree representation

Tree oriented programming tools

should facilitate: Defining trees Parsing Transforming Prettyprinting

Mainstream approach totree oriented programming

Defining

trees Parsing Transforming Prettyprinting

libraryclever hackingpreprocessorOO programming language

Our approach totree oriented programming

Defining

trees Parsing Transforming Prettyprinting

library

librarypreprocessor

functional languageHaskell

This morning’s programme

A crash course inFunctional programming using Haskell

Defining trees in Haskell The parsing library Transforming trees

using the UU Attribute Grammar Compiler

Prettyprinting Epilogue: Research opportunities

Language evolution:Imperative & Functional

50 years ago

Haskell

Part I

A crash course inFunctional programming

using Haskell

Function definition

static int fac (int n){ int count, res; res = 1; for (count=1; count<=n; count++) res *= count; return res;}

fac n = product [1..n] Haskell

fac :: Int Int

Definition forms

Function

Constant

Operator

fac :: Int Intfac n = product [1..n]

pi :: Floatpi = 3.1415926535

( !^! ) :: Int Int Intn !^! k = fac n / (fac k * fac (n-k))

Case distinction with guards

abs :: Int Int

abs x = x= -x

| x>=0| x<0

“guards”

Case distinction with patterns

day :: Int Stringday 1 = “Monday”day 2 = “Tuesday”day 3 = “Wednesday”day 4 = “Thursday”day 5 = “Friday”day 6 = “Saturday”day 7 = “Sunday”

constantas formal

parameter!

Iteration

fac :: Int Int

fac n = 1=

| n==0| n>0

recursion

n * fac (n-1)

without using standard function

product

List:a built-in data structure

List: 0 or more values of the same type

“empty list” constant

“put in front” operator

Shorthand notation for lists

enumeration

[ 1, 3, 8, 2, 5]

[ 4 .. 9 ]

> 1 : [2, 3, 4][1, 2, 3, 4]

> 1 : [4..6][1, 4, 5, 6]

Functions on lists

sum :: [Int] Intsum [ ] = 0sum (x:xs) = x + sum xs

length :: [Int] Intlength [ ] = 0length (x:xs)= 1 + length xs

patterns recursion

Standard library of functions on lists

> null [ ]True

> [1,2] ++ [3,4,5][1, 2, 3, 4, 5]

> take 3 [2..10][2, 3, 4]

challenge:Define these functions, using pattern matching and recursion

Functions on lists

null [ ] = Truenull (x:xs) = False

[ ] ++ ys = ys(x:xs) ++ ys = x : (xs++ys)

take 0 xs = [ ]take n [ ] = [ ]take n (x:xs) = x : take (n-1) xs

null :: [a] Bool

(++) :: [a] [a] [a]

take :: Int [a] [a]

Polymorphic type

Type involving type variables

take :: Int [a] [a]

Why did it take10 years and5 versionsto put this in Java?

Functions as parameter

Apply a function to allelements of a list

> map fac [1, 2, 3, 4, 5][1, 2, 6, 24, 120]

> map sqrt [1.0, 2.0, 3.0, 4.0][1.0, 1.41421, 1.73205, 2.0]

> map even [1 .. 6][False, True, False, True, False, True]

Challenge

What is the type of map ?

What is the definition of map ?

map ::

[a] (ab) [b]

map f [ ] =map f (x:xs)=

[ ]map f xs

Another list function: filter

Selects list elements thatfulfill a given predicate

filter :: (aBool) [a] [a]filter p [ ] =filter p (x:xs) =

[ ]x : filter p xs| p x

| True = filter p xs

> filter even [1 .. 10][2, 4, 6, 8, 10]

Higher order functions:repetitive pattern? Parameterize!

product :: [Int] Int product [ ] =product (x:xs)

1product xsx *

and :: [Bool] Bool and [ ] =and (x:xs)=

Trueand xsx

&&sum :: [Int] Int sum [ ] =sum (x:xs)

0sum xsx +

Universal list traversal: foldr

foldr :: [a] a

foldr (#) e [ ] =foldr (#) e (x:xs)=

efoldr (#) e xsx #

(aaa) a

combining function start value

foldr :: (abb) b [a] b

Partial parameterization

foldr is a generalizationof sum, product, and and ....

…thus sum, product, and andare special cases of foldr

product = foldr (*) 1and = foldr (&&) Truesum = foldr (+) 0or = foldr (||) False

Example: sorting (1/2)

insert :: a [a] [a]insert e [ ] = [ e ]insert e (x:xs)

| e x = e : x : xs| e x = x : insert e xs

isort :: [a] [a]isort [ ] = [ ]isort (x:xs) = insert x (isort xs)

isort = foldr insert [ ]

Example: sorting (2/2)

qsort :: [a] [a] [a]qsort [ ] = [ ]qsort (x:xs) = qsort (filter (<x) xs) ++ [x] ++ qsort (filter (x) xs)

(Why don’t they teach itlike that in thealgorithms course?)

Infinite lists

repeat :: a [a]repeat x = x : repeat x

> repeat 3[3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3

replicate :: Int a [a]replicate n x = take n (repeat x)

> concat (replicate 5 ”IPA ” )”IPA IPA IPA IPA IPA ”

Lazy evaluation

Parameter evaluation is postponeduntil they are really needed

Also for the (:) operatorso only the part of the listthat is needed is evaluated

Generic iteration

iterate :: (aa) a [a]iterate f x = x : iterate f (f x)

> iterate (+1) 3[3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20

Convenient notations(borrowed from mathematics)

Lambda abstraction

List comprehension

\x x*x

[ x*y | x [1..10] , even x , y [1..x] ]

for creating anonymous functionsmore intuitive thanequivalent expression using map , filter & concat

Part II

Defining treesin Haskell

Binary trees

15 29103

1 6 11

18 26 34

14with internallabels

How would you do thisin Java/C++/C# etc?

The OO approach to trees

class Tree{ private Tree left, right;

private int value;

// constructorpublic Tree(Tree al, Tree ar, int av){ left = al; right=ar; value=av; }

// leafs are represented as null}

The OO approach to trees:binary trees with external labels

class Tree {// empty superclass

}class Leaf extends Tree {

int value}class Node extends Tree {

Tree left,right}

Functional approach to trees

I need a polymorphic type and constructor functions

Leaf :: a Tree aNode :: Tree a Tree a Tree a

Tree a

data Tree a= Leaf a | Node (Tree a) (Tree a)

Haskell notation:

Example

Data types needed in a compilerfor a simple imperative language

data Stat= Assign Name Expr | Call Name [Expr] | If Expr Stat | While Expr Stat | Block [Stat]

data Expr= Const Int | Var Name | Form Expr Op Expr

type Name = Stringdata Op = Plus | Min | Mul | Div

Functions on trees

In analogy to functions on lists

we can define functions on trees

length :: [a] Intlength [ ] = 0length (x:xs)= 1 + length xs

size :: Tree a Intsize (Leaf v) = 1size (Node lef rit) = size lef + size rit

Challenge: write tree functions

elem tests element occurrence in tree

front collects all values in a list

elem :: a Tree a Boolelem x (Leaf y) = x==yelem x (Node lef rit) = elem x lef || elem x rit

front :: Tree a [a]front (Leaf y) = [ y ]front (Node lef rit) = front lef ++ front rit

A generic tree traversal

In analogy to foldr on lists

we can define foldT on trees

foldr ::

(abb) -- for (:)

b -- for [ ]

foldT ::

Tree a b

(ab) -- for Leaf (bbb) -- for Node

Challenge: rewrite elem and front using foldT

foldT ::

Tree a b

elem x (Leaf y) = x==yelem x (Node lef rit) = elem x lef || elem x rit

front (Leaf y) = [ y ]front (Node lef rit) = front lef ++ front rit

elem x = foldT (==x) (||)

front = foldT (\y[y]) (++)front = foldT ( :[] ) (++)

Part III

A HaskellParsing library

Approaches to parsing

Mainstream approach (imperative) Special notation for grammars Preprocessor translates grammar to C/Java/…

Our approach (functional) Library of grammar-manipulating functions

-YACC (Yet Another Compiler Compiler) -ANTLR (ANother Tool for Language Recognition)

ANTLR generates Javafrom grammarExpr : Term

( PLUS Term | MINUS Term ) *

;Term : NUMBER

| OPEN Expr CLOSE ;

public void expr (){ term (); loop1: while (true) { switch(sym) { case PLUS: match(PLUS); term (); break; case MINUS: match(MINUS); term (); break; default: break loop1; } }}public void term(){ switch(sym) { case INT: match(NUMBER); break; case LPAREN: match(OPEN); expr (); match(CLOSE); break; default: throw new ParseError(); }}

ANTLR: adding semantics

: Term ( PLUS Term| MINUS Term) *

;Term: NUMBER| OPEN Expr CLOSE ;

returns [int x=0]{ int y; }

returns [int x=0]

{ x += y; }{ x –= y; }

n: { x = str2int(n.getText(); }

{ $$ += $1; }

Yacc notation:

A Haskell parsing library

Building blocks

Combinators

type Parser

symbol :: a Parser satisfy :: (aBool) Parser

() :: Parser Parser Parser() :: Parser Parser Parser

epsilon :: Parser

A Haskell parsing library

Building blocks

Combinators

symbol :: a Parser satisfy :: (aBool) Parser

() :: Parser Parser Parser() :: Parser Parser Parser

type Parser a b

symbol :: a Parser a asatisfy :: (aBool) Parser a a

() :: Parser a b Parser a b Parser a b() :: Parser a b Parser a c Parser a (b,c)() :: Parser a b Parser a b Parser a b() :: Parser a b Parser a c Parser a (b,c)(®) :: (bc) Parser a b Parser a c

start :: Parser a b [a] b

epsilon :: Parserepsilon :: Parser a ()

Domainspecific Combinator Language vs. Library

New notation and semantics

Preprocessing phase

What you gotis all you get

Familiar syntax, just new functions

‘Link & go’ Extensible at will

using existing function abstraction mechnism

Expression parseropen = symbol ‘(’close = symbol ‘)’plus = symbol ‘+’minus = symbol ‘–’

expr = term (plusminus) expr term

term = number open expr close

data Tree = Leaf Int | Node Tree Op Treetype Op = Char

middle

expr, term :: Parser Char Tree

where middle (x,(y,z)) = y

Example of extensibility

Shorthand

Parameterized shorthand

New combinators

open = symbol ‘(’close = symbol ‘)’

pack :: Parser a b Parser a bpack p = open p closemiddle

many :: Parser a b Parser a [b]

The real type of ()

() :: Parser a b Parser a b Parser a b() :: Parser a b Parser a c Parser a (b,c)(®) :: (bc) Parser a b Parser a c

How to combine b

and c ?

() :: Parser a b Parser a c (bcd) Parser a d

() :: Parser a (cd) Parser a c Parser a d

pack p = open p closemiddle where middle x y z = y

Another parser example;design of a new combinator

many :: Parser a b Parser a [b]

many p = p many p epsilon

(\b bs b:bs)

(\e [ ])

many p = p many p succeed [ ]

Challenge:parser combinator design

many :: Parser a b Parser a [b]many1 :: Parser a b Parser a [b]sequence :: [ Parser a b ] Parser a [b]

EBNF * EBNF + Beyond EBNF

many1 p = p many p

sequence [ ] =sequence (p:ps) =

succeed [ ](:) p sequence

pssequence = foldr

f (succeed [])where f p r = (:) p r

More parser combinators

sequence :: [ Parser a b ] Parser a [b]choice :: [ Parser a b ] Parser a [b]listOf :: Parser a b Parser a s Parser a [b]

choice = foldr () fail

listOf p s = p

many ( s p )

separator

(\s b b)

chain :: Parser a b Parser a (bbb) Parser a b

Example: Expressions with precedence

Method call

Parser shouldresolve

precedences

Parser for Expressions(with precedence)

expr = chain term (symbol ‘+’)

term = chain fact (symbol ‘*’)

fact = number pack expr

(\o(:+:))

(\o(:*:))

( (\o(:–:)) (symbol ‘–’))( (\o(:/:)) (symbol ‘/’))

Var name Fun name pack (listOf expr

(symbol ‘,’) )

A programmers’ reflex:Generalize!

expr = chain term ( … (:+:)…‘+’ … … (:–:)…‘–’ …)

term = chain fact ( … (:*:)…‘*’ … … (:/:)…‘/’ …)

fact = basicCases pack expr

gen ops next= chain next ( choice …ops…

Expression parser(many precedence levels)

expr = gen ops1 term1term1= gen ops2 term2term2= gen ops3 term3term3= gen ops4 term4term4= gen ops5 factfact = basicCases

pack expr

expr = foldr gen fact [ops5,ops4,ops3,ops2,ops1]

gen ops next= chain next ( choice …ops…

expr = gen ops1 term1term1= gen ops2 term2term2= gen ops3 term3term3= gen ops4 term4term4= gen ops5 fact

Library implementation

type Parser = String X

type Parser b = String b

polymorphicresult type

type Parser b = String (b, String) rest string

type Parser a b = [a] (b, [a]) polymorphic

alfabet

type Parser a b = [a] [ (b, [a]) ] list of successesfor ambiguity

Library implementation

() :: Parser a b Parser a b Parser a b

(p q) xs = p xs ++ q xs

() :: Parser a (cd) Parser a c Parser a d

(p q) xs =p xsq ys

(f,ys) [|,]

(c,zs)

( , )f c zs

() :: (bc) Parser a b Parser a c(f p) xs =

p xs(b,ys) [|]

( f b , ys )

Part IV

Techniques forTransforming trees

Data structure traversal

In analogy to foldr on lists

we can define foldT on binary trees

foldr ::

(abb) -- for (:)

b -- for [ ]

foldT ::

Tree a b

Traversal of Expressions

data Expr= Add Expr Expr| Mul Expr Expr| Con Int

foldE ::

Expr b

(bbb) -- for Add

(bbb) -- for Mul(Int b) -- for Con

type ESem b = ( b b b , b b b , Int b )

Traversal of Expressions

foldE :: ESem b Expr b

foldE (a,m,c) = f where f (Add e1 e2) = a (f e1) (f e2) f (Mul e1 e2) = m(f e1) (f e2) f (Con n) = c n

Using and defining Semantics

evalExpr :: Expr IntevalExpr = foldE evalSem

evalSem :: ESem IntevalSem = ( (+) , (*) , id )

Syntax and Semantics

“ 3 + 4 * 5 ”

Add (Con 3) (Mul (Con 4) (Con 5))

parseExpr

evalExpr

= start p where p = ………

= foldE s where s = (…,…,…,…)

Multiple Semantics

“ 3 + 4 * 5 ”

evalExpr compileExpr

Push 3Push 4Push 5Apply (*)Apply (+)

runCode

parseExpr

:: String

:: Expr

:: Int :: Code

= foldE s where s = (…,…,…,…) s::ESem Int

= foldE s where s = (…,…,…,…) s::ESem Code

A virtual machine

What is “machine code” ?

What is an “instruction” ?

type Code = [ Instr ]

data Instr = Push Int | Apply (IntIntInt)

Compiler generates Code

evalExpr :: Expr IntevalExpr = foldE evalSem where evalSem :: ESem Int evalSem = ( (+) , (*) , id )

compExpr :: Expr CodecompExpr = foldE compSem where compSem :: ESem Code compSem = ( add , mul , con )

mul :: Code Code Codemul c1 c2 = c1 ++ c2 ++ [Apply (*)]con n = [ Push n ]

Compiler correctness

evalExpr

“ 3 + 4 * 5 ”

compileExpr

Push 3Push 4Push 5Apply (*)Apply (+)

runCode

parseExpr

runCode (compileExpr e)=

evalExpr e

runCode:virtual machine specification

run :: Code Stack Stackrun [ ] stack = stackrun (instr:rest) stack = exec instr stackrun rest ( )

exec :: Instr Stack Stackexec (Push x) stack = x : stackexec (Apply f) (x:y:stack) = f x y : stack

runCode :: Code IntrunCode prog = run prog [ ]hd ( )

Extending the example:variables and local def’s

data Expr= Add Expr Expr| Mul Expr Expr| Con Int| Var String| Def String Expr Expr

type ESem b = ( b b b , b b b , Int b , String b , String bbb )

evalExpr :: Expr IntevalExpr = foldE evalSem where evalSem :: ESem Int evalSem = ( add , mul , con ), var, def )

Any semanticsfor Expressionadd :: b b b add x y =

mul :: b b b mul x y =

con :: Int b con n =

var :: String b var x =

def :: String b b b def x d b =

Evaluation semanticsfor Expressionadd :: b b b add x y =

Int Int (EnvInt)Int

Int IntInt

Evaluation semanticsfor Expressionadd :: b b b add x y =

Int Int (EnvInt)Int

Int IntInt

lookup e x\e (EnvInt)

(EnvInt)

(EnvInt

Evaluation semantics for Expressionadd :: b b b add x y =

Int Int (EnvInt)Int

IntInt

lookup e x\e

(EnvInt)

(EnvInt)(EnvInt)

(EnvInt) (EnvInt)

(EnvInt)(EnvInt)

b e((x,d e) : )

Extending the virtual machine

What is “machine code” ?

What is an “instruction” ?

type Code = [ Instr ]

data Instr = Push Int | Apply (IntIntInt)

data Instr = Push Int | Apply (IntIntInt) | Load Adress | Store Adress

Compilation semantics for Expressionadd :: b b b add x y =

(EnvCode) (EnvCode) Env Code

Env Code

(EnvCode) (EnvCode) Env Code

x e ++ y e ++ [Apply (+)]

x e ++ y e ++ [Apply (*)]

[Push n]

[Load (lookup e x)]

d e++ b e((x,a) : )[Store a]++

where a = length e

Language: syntax and semantics

type ESem b = ( b b b , b b b , Int b , String b , String bb b )

compSem :: ESemcompSem = (f1, f2, f3, f4, f5) where ……

(EnvCode)

compile t = foldE compSem t [ ]

Language: syntax and semantics

type ESem b = ( b b b , b b b , Int b , String b , String bb b )

compSem :: ESemcompSem = (f1, f2, f3, f4, f5) where ……

(EnvCode)

compile t = foldE compSem t [ ]

type ESem b c = ( ( b b b , b b b , Int b , String b ) , ( String b c , b c c , b c c c , [ c ] c ) )

Code(EnvCode)compSem = ((f1, f2, f3, f4), (f5, f6, f7, f8)) ……

Real-size example

data Module = ……data Class = ……data Method = ……data Stat = ……data Expr = ……data Decl = ……data Type = ……

type ESem a b c d e f = ( (…,…,…) , (…,...) , (…,…,…,…,…,…) , …

compSem :: ESem

compSem = (…dozens of functions…) ……

(…… ……)(…… ……)(…… ……)(…… ……)(…… ……)(…… ……)(…… ……)

Attributesthat are passed

top-down

Attributesthat are generated

bottom-up

Tree semantics

data Expr= Add Expr Expr| Var String| …

codeSem =( \ a b \ e a e ++ b e ++ [Apply (+)], \ x \ e [Load (lookup e x)], ……

ATTR Expr inh e: Env syn c: Code

Explicit names for fields and attributes

DATA Expr= Add a: Expr b: Expr| Var x: String| …

generated by Attribute Grammar

SEM Expr| Add this.code = a.code ++ b.code ++ [Apply (+)]

a.e = this.eb.e = this.e

| Var this.code = [Load (lookup e x)]

Attribute value equationsinstead of functions

UU-AGCAttribute Grammar Compiler

Preprocessor to Haskell Takes:

Attribute grammar Attribute value definitions

Generates: datatype, fold function and Sem type Semantic function (many-tuple of

functions) Automatically inserts trival def’s

a.e = this.e

UU-AGCAttribute Grammar Compiler

Advantages: Very intuitive view on trees

no need to handle 27-tuples of functions

Still full Haskell power in attribute def’s Attribute def’s can be arranged modularly No need to write trivial attribute def’s

Disadvantages: Separate preprocessing phase

Part IV

Pretty printing

Inputtext

Outputtexttransform prettyprintparse

internal tree representation

SEM Stat| Assign this.code = …| While this.code = …| Block this.code = …

Prettyprinting is justanother tree transformation

Example:transformation from Stat to String

SEM Stat| Assign this.code = x.code ++ “=” ++ e.code ++ “;”| While this.code = “while (” ++ e.code ++ “)”++ s.code| Block this.code = “{” ++ body.code ++ “}”

ATTR Expr Stat [Stat] syn code: String

DATA Stat= Assign a: Expr b: Expr| While e: Expr s: Stat| Block body: [Stat]

SEM Stat| While s.indent = this.indent + 4

inh indent: Int

But howto handlenewlines &indentation?

A combinator libraryfor prettyprinting

Type Building block Combinators

Observer

type PPDoc

text :: String PPDoc

(>|<) :: PPDoc PPDoc PPDoc(>–<) :: PPDoc PPDoc PPDocindent :: Int PPDoc PPDoc

render :: Int PPDoc String

Epilogue

Research opportunities

Research opportunities (1/4)

Parsing library: API-compatible to naïve library, but With error-recovery etc. Optimized

Implemented using the “Attribute Grammar” way of thinking

UU - Attribute Grammar Compiler More automatical insertions Pass analysis optimisation

A real large compiler (for Haskell) 6 intermediate datatypes 5 transformations + many more

Learn about software engineering aspectsof our methodology

Reasearch opportunities (4/4)

Generate as much aspossible with preprocessors Attribute Grammar Compiler Shuffle

extract multiple views & docsfrom the same source

Rulergenerate proof ruleschecked & executable

tree oriented programming jeroen fokker. tree oriented programming nmany problems are like: input...

Documents

design patterns in object-oriented abap - amazon s3 · pdf...

from c and java to...

b+ tree definition b+ tree properties b+ tree searching ...

effective planning and control of large projects-using...

content-oriented listening types of listening styles...

cmsc 132: object-oriented programming ii · 2-3-4 tree...

balanced tree(avl tree,red black tree)

object-oriented...

reframing deficit-oriented questions deficit-oriented

the object-oriented design of the expression tree...

collection--oriented oriented collection-oriented ... ·...

service service - - oriented oriented architecture

chick-fil-a fircrestdial 811 callbefore you dig. know...

tree oriented data analysis - arxiv · pdf file1.5 other...

tree inclusion, signatures, and evaluation of path-oriented...

learning from experience - fraunhofer...nmany tools publicly...

institut für informatik -...

preventing scope creep. work breakdown sructure (wbs) aa...

balanced tree (avl tree & red-black tree)

service oriented architecture service oriented architecture