Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

1

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy EPOS and Myrinet: Effective Communication Support for

Parallel Applications Running on Clusters of Workstations

Antônio Augusto FröhlichGilles Pokam Tientcheu

Wolfgang Schöder-Preikschat

mailto:guto@first.gmd.dehttp://www.first.gmd.de/~guto

May 2000

2

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Outline Introduction

Application-orientation systems Component-based systems

EPOS Design Framework Generation Tools

Conclusion

3

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

high performanceX

all purpose, global, generic

Introduction

There is no best “general” solution Each (class of) application has particular demands

4

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Applicaiton-orientation Each application deserves its own run-time

support system Set of applications that will run on top of the

operating system and their requirements Isn’t known until run-time => dynamic adaptation Can be determined before run-time => static

configurationParallel and embedded applications

Adaptability and configurability Component-based?

5

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Component-orientation

componentgranularity

many simple components customizability

reusabilityperformance“a queue link”

“a file system”usability

maintainabilitya few complex components

6

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gyThe Gap to Applications

(Motivation for EPOS)

componentgranularity

many simple componentsWhat is this component for?

What is the difference to that other one?How do I use it?

Do I have to take this junk as well?The component I need is not there!a few complex components

7

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gyEPOS: Embedded Parallel

Operating System EPOS operating system

Application-driven assemblage of system components

EPOS components Statically configurable, application ready system

abstractions EPOS and the real world

Intelligent visual tools for configuration Invisibility

Standard libraries (Posix files, libc, libstdc++, libm)Standard APIs (Posix threads, MPI)

8

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

System Abstraction

system micro-components

EPOS System Abstractions Application ready

components Independent from

execution scenario Examples:

A thread on a given scheduling policy

NOT a thread for one processor, or for multi-tasking

9

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Scenario Adapters

EPOS Scenario Adapters Adapt existing system

abstractions for a given execution scenario

Adapt system micro-components to grant the semantics dictated by a given execution scenario

Examples: An SMP adapter A secure remote

invocation adapter

10

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

task

memory segment

thread

file

synchronizer

communicator

Inflated Interfaces

EPOS Inflated Interfaces Export system abstractions

to applications Well-known to application

programmers Comprehensive Promote requirement

analysis Examples:

thread communicator synchronizer

Not a fat interface if intersection => subset

11

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

realization

<<interface>>

partially realize selectively realize

key

realization 1

<<interface>>

realization 3realization 2

1

2

3

Partial and Selective Realization Relationships

System configuration is basically restricted to the setting of selective realize keys

12

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Wrapped No allocation or sharing

control No cross-domain

invocation Always embedded in

the application

Controlled Allocation and sharing

control is possible Cross-domain invocation

is possible Embedded in the

application or packed in a kernel

EPOS Framework Static metaprogrammed

Metaprograms run at compile-time (Almost) no run-time overhead Implemented as C++ templates

System abstraction implementation

13

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

inflated interfacescenario adapter

<<select>>

wrapper realization<<select>>

Application

EPOS FrameworkWrapped System Abstraction

14

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

basic stub

message exchange stubprocedure call stubInflated interface

<<select>>stub

<<select>>server

<<select>>realization

basic server

message exchange serverprocedure call server

handle

<<select>>id

<<select>>message

<<select>>id

state

Application

System

EPOS FrameworkControlled System Abstraction

<select>>scenario adapter

15

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Automatic Generation Application refers to inflated interfaces Requirement analyzer parses the application

searching for inflated interfaces references Which interfaces are referred to? How they are referred to?

Expert system select the realizations that better match the referred inflated interfaces

An application-oriented operating system is compiled

16

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

syntacticanalyzer

tasks

segments

threadsmailboxes

mutexfiles

complex application programcode = new Segment(buffer, size);task = new Task(code, data);thread = new Thread(task, &entry_func, priority, SUSPENDED, arguments);mutex->entry();Mailbox mailbox;mailbox >> message; File file(name);file << mailbox;

Requirement Analysis

17

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Requirement Analysis

flat memorylinks

syntacticanalyzer

real application program (MPI)Channel link(destination);link << mesage;

18

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Context Information We need context information

new Thread(task, &func) => location?mailbox << message => protocol? Absence of Task => single-task?

Buildup databases Scenario dependencies Target machine description

19

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Execution Scenario

execution scenario

+ =tasks

segments

threadsmailboxes

monitorsfiles

required Inflated interfaces

20

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Application-oriented EPOS

application

interfaces

scenario adapters

system micro-components system abstractions

21

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Conclusions EPOS doesn’t grant the tailored OS to be optimal

When several realizations fulfill the requirements, the selection is arbitrary

Profiling could help EPOS framework overhead is close to zero Tools are being implemented System abstraction repository is growing First real applications are being negotiate

22

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Titel:send.epsErstellt von:fig2dev Version 3.2 Patchlevel 1Vorschau:Diese EPS-Grafik wurde nicht gespeichertmit einer enthaltenen Vorschau.Kommentar:Diese EPS-Grafik wird an einenPostScript-Drucker gedruckt, aber nichtan andere Druckertypen.

23

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Titel:bandwidth.epsErstellt von:gnuplot 3.7 patchlevel 0.2Vorschau:Diese EPS-Grafik wurde nicht gespeichertmit einer enthaltenen Vorschau.Kommentar:Diese EPS-Grafik wird an einenPostScript-Drucker gedruckt, aber nichtan andere Druckertypen.

24

Germ

an N

atio

nal R

esea

rch

Cent

er fo

r Inf

orm

atio

n Te

chno

logy

Rese

arch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Titel:latency.epsErstellt von:gnuplot 3.7 patchlevel 0.2Vorschau:Diese EPS-Grafik wurde nicht gespeichertmit einer enthaltenen Vorschau.Kommentar:Diese EPS-Grafik wird an einenPostScript-Drucker gedruckt, aber nichtan andere Druckertypen.