implementation: charm++ orion sky lawlor [email protected]
TRANSCRIPT
![Page 2: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/2.jpg)
Background & Roadmap
Your code sits on Charm++: Interface Translator
.ci file, CProxy_*, parameter marshalling
Arrays (Orion++) Reductions, broadcasts, migrations, pup
Groups and Chares (``Milind++) Basic communication calls atop Converse
Basics (++) Registration, readonlies, messages
![Page 3: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/3.jpg)
Where is it? Charm++ core source code in
charm/src/Common/ck-core/ Gets soft-linked over and made in
charm/net-linux/tmp/ Headers in charm/net-linux/include/ Code gets linked into the library
charm/net-linux/lib/libck.a This all gets used when you pass
"-language charm++" to charmc
![Page 4: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/4.jpg)
Basics: We need to be able to call some
C++ method bar on user object foo
But the core`s never heard of foo Solution: register a function
pointer to a "call-method" function:
void _call_foo_bar(void *msg,foo *obj) {obj->bar(msg);
}
...at startup...int barEpIdx=
CkRegisterEp(&_call_foo_bar,..);
![Page 5: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/5.jpg)
Registration & Startup ck-core/register.h .C (300 lines) At startup, each module registers
its methods, which end up in a big entry method table
An entry point's index in this table is its int "epIdx", used everywhere
Translator writes this registration code (the call-method function, "__register" function) in .def file
![Page 6: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/6.jpg)
Groups & Chares (Milind++)
charm.h, ck.C (1000 lines) Lowest layer of Charm++ CkCreateChare, CkCreateGroup CkSendMsg, CkSendMsgBranch Quite simple, thin wrapper on
Converse Very stable-rarely changes Callable from C (!)
![Page 7: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/7.jpg)
CkSendMsg(int epIdx, void *msg,CkChareID
*to) envelope *env = UsrToEnv(msg);
Finds Charm++ msg fields (see envelope.h)
env->setMsgtype(ForChareMsg);
env->setObjPtr(to->objPtr);
env->setEpIdx(epIdx);
env->setSrcPe(CkMyPe());
CmiSetHandler(env, _charmHandlerIdx);
CldEnqueue(to->onPE, env, _infoIdx);
![Page 8: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/8.jpg)
CkSendMsg: Clutter Various unrelated calls clutter up the
Charm++ core (FIXME?) Keep track of message sends; give
"Message being re-sent..." diagnostic _CHECK_USED(env);
_SET_USED(env, 1);
Trace message sends for projections _TRACE_CREATION_1(env);
Notify quiescence detection of send CpvAccess(_qd)->create();
![Page 9: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/9.jpg)
Arrays (Orion++) Many interesting features:
Arbitrary indexed collection of Chares Connects to load balancer Supports migration, broadcasts, reductions Not even clear some operations even possible
But much more complicated Highly fragmented, er, factored code
Array Manager, Location Manager, Reduction Manager, Broadcaster, Reducer
![Page 10: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/10.jpg)
Arrays Diagram
CkReductionMgr--reductions
CkLocMgr--migration
CkArray-- coordinator
ArrayElementArrayElementArrayElementArrayElement-
-user's work
![Page 11: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/11.jpg)
CkReductionMgr ckreduction.h .C .ci (1300 lines) Accept contributions from local
members of a migrating set of reduction "contributors"
Implemented as Group Actually "IrrGroup", since normal groups
inherit from CkReductionMgr.
Exactly one per array; but zero dependence on arrays
Arrays are just a client of CkReductionMgr
![Page 12: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/12.jpg)
Location-- CkLocMgr ck-core/cklocation.h .C .ci (2000) Keep track of location of migrating
elements Handles messaging and migration Each array has a location manager
But a location manager may have several "bound" arrays, which will migrate together
Implemented as a group
![Page 13: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/13.jpg)
Array Manager-- CkArray
ck-core/ckarray.h .C .ci (1300 lines) Keeps track of a set of local array
elements Implemented as a group Glue that connects pieces of array
system together: Inherits from CkReductionMgr Implements CkArrMgr (for location mgr) Interfaces with proxies Controls ArrayReducer, ArrayBroadcaster
![Page 14: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/14.jpg)
Pup util/pup.h .C (1000 lines) Used to migrate, parameter marshall Heavy C++: overload the | operator "Easy to use, tough to understand" Bottom line: (user view)
Say "p|x;" for all your x's.
Bottom line: (system view) We get a "void *" for each of the user`s x.
![Page 15: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/15.jpg)
Translator xlat-i/xi-symbol.h .C (3000 lines) Turn .ci files into .decl/.def files
Standalone program "charmxi" Tokenize .ci with lexx: xi-scan.l Parse .ci with yacc: xi-grammar.y Turn parsed structures into .decl/.def
Almost completely routines named "gen*"-- code-generating routines
E.g., "Entry::genDecls"-- generate code in .decl file for an entry method
![Page 16: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/16.jpg)
Not Covered Startup & Shutdown (init.C) Quiescence detection (qd.C) Dynamic load balancer (ck-ldb/) Callbacks, futures (for sync
methods), debug, TeMPO Libraries (charm/src/Common/lib/)
TCharm, FEM, AMPI, MBlock, AMR, comm, ...
Details! Read manuals, headers, our minds...
![Page 17: Implementation: Charm++ Orion Sky Lawlor olawlor@acm.org](https://reader033.vdocuments.net/reader033/viewer/2022051417/5697c0281a28abf838cd70b1/html5/thumbnails/17.jpg)
Conclusion Interface Translator
.ci file, CProxy_*, parameter marshalling
Arrays (Orion++) Reductions, broadcasts, migrations, pup
Groups and Chares (``Milind++) Basic communication calls atop Converse
Basics (++) Registration, readonlies, messages