now you c it. now you don’t!

Now You C It. Now You Don’t!

Rob EnnalsIntel Research Cambridge

1/16/06 Now you C it. Now you Don't! (Dec Langs for Multicore)2

The World has Changed

Then Now

Parallelism an exotic curiosity

Slow inter-chip communication

Parallel chunks must be large

Uni-processor performanceincreasing rapidly

Not worthwhile for languages, tools, or programmers to payattention to parallel hardware

Multicore chips the norm

Fast on-die communication

Parallel chunks can be tiny

Uni-processor performance stalling, multicore storming

Essential that languages,tools and programmers

pay attention to multicore


An Opportunity for Declarative Languages

Parallelisability is more important than straight line performance– Number of cores set to double every 18 months

Declarative languages make parallelism easier– Easier to isolate parallel threads (pure functions, effect systems, etc)– Easier to express parallel algorithms (closures, combinators, etc)

Imperative Declarative

Very fast in a single core

Hard to express parallelism

Widely known and used

Often slower on a single core

Easier to express parallelism

Little known or used


A Problem: Language Switching Costs

Much important software is currently written in C– Even if not the most lines of code, probably most of the cycles

Moving to a new language incurs high switching costs– Programmers, tools, libraries, and existing code, all tied to C

C

Programmers

Trust

Libraries

Existing Code Tools


A Solution: Lossless Round Tripping

Jekyll is a high level functional programming language– Featuring most of the features of Haskell + more

Jekyll can be translated losslessly to and from C– Preserving layout, formatting, comments, everything– C code is readable and editable

C File

C File

Jekyll File

Jekyll File

C Programmer Jekyll Programmer


Another View of C

Authoritative source code can stay as C

But programmers and tools can also view it as Jekyll

C Programmers need not know Jekyll is even being used.

Repository

C File

C File

Jekyll File

C File

Jekyll Programmer or Tool

C Programmer or Tool


Switching Costs are Reduced

Programmers and Tools can still use the C version.

Existing C code can stay in C– Although there may be benefit to be had from modifying it

If Jekyll ceases to be maintained, just use the C

Jekyll

C Programmers

C Trust

C Libraries

Existing C Code C Tools


Jekyll Features

Parallel features still in progress. Other features largely implemented.

Use of unsafe features causes a warning unless marked as “unsafe”

All of C

Unsafe FeaturesImperative FeaturesLow-Level Features

C TypesC ExpressionsPre-processor

Most of Haskell

Algebraic TypesType Classes

Lambda ExpressionsPattern Matching

Generic TypesType SafetyOptional GC

Parallel

Parallel CombinatorsEffect TypingAtomic Blocks

Jekyll


Jekyll Parallel Features (in progress)

Parallel Combinators - express parallelism at a higher level– Use high level concepts such as parallel map, reduce, pipeline, etc– Easily write new combinators– Write tools that understand and manipulate combinators– Relies on Jekyll’s lambda expressions

Effect Typing - ensure threads are cleanly separated– Use linear types + regions + effect types – Determine what a thread can touch– Relies on Jekyll’s type-safe foundation

Atomic Blocks - avoid the mess of locks– Simple, blocking semantics (no STM required, can be just a global lock)– Translate to locking, or use hardware


Encoding Jekyll Features into C - Example

List<int>* multlist(List<int>* l, int x){return par_map (l) { int* n: ret new (*n) * x;};

}

_localfunenv struct multlist_lam_env {int *x};

_localfun int* multlist_lam(struct multlist_lam_env* _cenv, int* n){_temp int *_tmp;_tmp0 = (int*)GC_malloc(sizeof(int));(*_tmp0) = (*n) * *_cenv->x;return _tmp;

}

List _p(int)* multlist(List _p(int)* l, int x){_temp struct multlist_lam_env _f0_env = {&x};return List_par_map(_env NULL,l,_localfun (*(*)(void*,a*))multlist_lam);

}


Jekyll Features Encoded using C Macros

Ignored by C compilers

Tell the Jekyll translator when Jekyll features are being used

All macros are very simple (most are defined to nothing)

#define unsafe /* nothing */#define _fwd /* nothing */#define _temp /* nothing */#define _localfun /* nothing */#define _localfunenv /* nothing */#define _env /* nothing */…

Jekyll_1.h


A Language with Two Syntaxes

Translation allow Jekyll to sit on both sides of the fence

Jekyll with transparent to C compilers, but also has an elegant syntax

Compatible with C Tools

Understood by C Programmers

Compatible with existing code

Annotated C Syntax Jekyll Syntax

Elegant

Concise

Better fit with new features

Translation


Common Syntax Tree Representation

ASTJekyllFile

CFile

Parse Parse

PrintPrint

Check

Transform


Lossless Translation by Twinned Printing

Every Jekyll token is twinned with a C token

Twinned tokens always have the same whitespace– Thus allowing whitespace to be preserved during translation

Some C tokens may be un-twinned (see next slide)– Some Jekyll-only features need more C tokens than Jekyll tokens

AST

C Tokens

Jekyll Tokens

Twins


An Issue: Untwinned C tokens

Problem:

Whitespace is NOT preserved for untwinned C tokens

But:

Untwinned tokens only appear when Jekyll-only features are used.

Whitespace is only lost when a C programmer edits such code.

Thus this is ok, since:

All existing C code is unaffected.

Jekyll -> C -> Jekyll is always lossless

Only edited lines of source code will change.

The programmer is warned in such circumstances.


Demo


Conclusions

• Multicore is a great opportunity for declarative languages

• But many developers are tied into C

• Jekyll overcomes switching costs through lossless translation

Download Jekyll now:

http://jekyllc.sf.net

now you c it. now you don’t!

Documents

c version

existing code

parallel map

parallel hardwaremulticore

everythingc code

new combinatorswrite

lines of code

use of unsafe features