how software architecture can make an application-friendly internet

Pamela Zave

AT&T Laboratories—Research

Florham Park, New Jersey, USA

HOW SOFTWARE ARCHITECTURE

CAN MAKE AN

APPLICATION-FRIENDLY INTERNET

ABOUT AT&T RESEARCH (www.research.att.com)

we are a direct descendantof Bell Labs

about 200 researchers

many of us live in New York Cityand travel to the lab by train

we hire new Ph.D.s every year

HOW SOFTWARE ARCHITECTURECAN MAKE AN APPLICATION-FRIENDLY INTERNET:OUTLINE

THE STATE OF INTERNET ARCHITECTURE

IDEAS THAT MAKE PROGRESS POSSIBLE

EXAMPLE: MOBILITY

A RESEARCH AGENDA

1

2

3

4

THE "CLASSIC" INTERNET ARCHITECTURE

THE LAYERS

Each layer is universal and has adifferent function.

THE PHILOSOPHY

applications

transport (TCP,UDP)

network (IP)

link

physical

Internetcore

cooperation among trusted parties(no security)

best-effort service (no reliability orperformance guarantees)

end-to-end transparency (no built-inservers)

designed to empower users andencourage innovation

despite its limitations

(or maybe because of them and the resulting simplicity),

this architecture has succeeded beyond anyone’s wildest dreams

[Clark 88]

THE AGE OF THE WEB (1993 - NOW)

private subnetpublic Internet

machines here have nopersistent public addresses;

NAT boxes play with addressesand ports

Firewalls and Network Address Translation (NAT) boxes provide security and address-space expansion.

default firewall allows noinitiation of communicationfrom this side, even if desirable

NAT boxes and firewalls . . .

. . . are so tightly intertwined with TCP and UDP,

. . . and vary so much across the Internet,

that unless they know about an application,it is unlikely to work.

in other words, onlyapplications built on HTTP

work reliablypeople speak of HTTP

as the new Internet core[Popa et al. 10]

lack ofseparation

of concerns

THE STATE OF INTERNET EVOLUTION [Handley 06]

INTERNET "OSSIFICATION"

there has been no importantchange in the transport layer (TCP/UDP) since 1988

" . . . technologies get deployed in the core of the Internet

when they solve an immediate problem

or when money can be made"

there has been no important change in the network layer (IP)since 1993

it is very difficult for an Internetservice provider to make moneywith improvements, because mosthave no effect until everyone elseadopts them

a crisis is the only way to get theglobal consensus required forreal change

THE AGE OF DIVERSITY (NOW - FUTURE)TO MEET THE NEEDS OF SOCIETY, THE FUTURE INTERNET MUST SUPPORT AMUCH WIDER RANGE OF . . .

[Clark et al. 05]

. . . APPLICATIONS

will replace current data, telecommunications, andbroadcast networks

real-time, peer-to-peer, and end-to-end transparent applications, notjust Web applications

. . . STAKEHOLDERS

all segments of society are stakeholders,their interests must be balanced

. . . RESOURCES

many kinds of wireless networks

delay-tolerant networks (“sneakernets”) incorporate very slow, verydiverse link technologies

. . . COMMUNICATION FUNCTIONS

many aspects of privacy andsecurity

support for mobility, multihoming,anycast, customized routing, etc.

. . . POLICIES

selective reliability guarantees

selective quality-of-service (QoS)guarantees

selective privacy and securityguarantees

the classic architecture andthe Age of the Web have created

a one-size-fits-all Internet thatcannot provide the needed diversity

WHEN THE NEED FOR DIVERSITY MEETS OSSIFICATIONa typical real packetshows (through headers)the use of these layers(simplified picture):

Ethernet

MPLS

IP + UDP

GTP

IP + IPSec

IP

HTTP

Cloud Services 15 or more load-balancingand routing algorithmsapply to this packet

each algorithm hasdifferent goals, and eachhas been analyzed mostlyin isolation

[Spatscheck 10]

link

network:

transport

middleware

TCP

these extralayers provideprivacy,security, QoS,billing,customizedrouting, etc.

THIS COMPLEXITY MAKES ITDIFFICULT TO MANAGERESOURCES WELL:

unanticipated effects ofbuffering on TCP arecausing widespreadperformance problems(high latency and jitter)

[Gettys 11]

THE COMPLEXITY CHALLENGE

HOW ARE WE DOING WITH

RESOURCE MANAGEMENT?

the future Internet must supportmuch greater diversity, at much greater scale

Not so well.

We have composition of toomany layers, and too little abilityto predict how they behave whencomposed.

HOW ARE WE DOING WITH

APPLICATIONS?

Also not well.

Because of the pervasive lack of separation of concerns, . . .

. . . a mechanism to solve a problem usually works only in a specialized environment, and

. . . mechanisms to solve problems tend to break each other.

As a result, it is much too difficult tobuild, deploy, and maintain applications.




EXAMPLE: MOBILITY

A RESEARCH AGENDA

1

2

3

4

EVOLUTION THROUGH LAYERS AND VIRTUALIZATION

LAYERS ARE THE MODULES OFNETWORK ARCHITECTURE

A NEW LAYER IS ALSO A "CLEAN SLATE"FOR DESIGN, . . .

. . . AND ITS VALUE IS ENHANCED BYVIRTUALIZATION

network resources(physical, link layers)

Internet core(network, transport

layers)

applications

OVERLAYS

OVERLAYS

slice ofresources

applications

NEW LAYERS NEW LAYERS

slice ofresources

applications

network resources

virtualization

can experiment safelywith new architecturalideas

in the end, there may be nouniversal Internet layers [Roscoe 2006]

we have seen that customoverlays are used frequently tomodify an ossified Internetarchitecture can get closer to the

resources thanInternet overlays can

AAA

A

A

A

WHAT IS A LAYER?

Using Day’s definitionof a layer [Day 08], each layer containsall the basic mechanismsof networking.

The definition is atemplate that canbe instantiateddifferently fordifferent purposes,scopes, and levels.

The definition makes clearhow overlays composein a hierarchy and whattheir relationships are. deeply rooted in the history and

practice of networking

draws the module boundary inexactly the right place

provides for diversity within acommon structure

THESE ARE NOT THE SAME AS CLASSICINTERNET LAYERS:

All layers have the same functions (in themost abstract sense) rather than havingdifferent functions in different layers.

THEY ARE WHAT WE NEED!

DAY'S DEFINITION OF A LAYER

AB

CD

E

Membership:the members areprocesses; each has aunique and persistentname from the namespace; enrollmentprotocol accepts andnames new members

Routing:any member can reach anyother through a path in thelayer; routing protocolspreads knowledge of linksand paths; forwarding protocoluses path knowledge

THERE ISSECURITY AND

RESOURCEMANAGEMENTTHROUGHOUT

Links:there is a link betweentwo member processesif both are registeredin the same lowerlayer

Registration:user processes in a higher layercan register their locations atmember processes on thesame machine; there is adirectory of registrations

Communication Service:the layer provides a specifiedservice for its users, e.g.,point-to-point sessionsB E

session(B,E)

PARTICULARAPPLICATIONLAYER

INTERNETCORELAYER

LOCALAREANETWORKS

networkattachments

applicationprocesses

machinesand devices

in this very populous layer, names areorganized hierarchically, based onadministration, geography, and topology

THE CLASSIC ARCHITECTURE ACCORDING TO DAY

within one layer, routing andforwarding (IP) can create lossand congestion; error controland flow control (TCP) managethese problems

routes lead to blocks ofnames, making routingon this scale possible

TO ACHIEVE DIVERSITY AND MANAGE COMPLEXITY

EACH APPLICATION SHOULD RUNON EXACTLY THE RIGHT STACK OFLAYERS . . .

application

all the necessary functions,appropriate policies,nothing superfluous

. . . BUT THIS WILL REQUIRE ALARGE NUMBER OF CUSTOMIZEDLAYERS!

THE KEY IS UNDERSTANDING THESOFTWARE ARCHITECTURE OFLAYERS . . .

a layer is adistributed software system

overlay

underlay underlay

. . . WELL ENOUGH SO THAT WE CANGENERATE THE NECESSARY(CORRECT, PREDICTABLE)SOFTWARE

[Loo et al. 05]

THE BEGINNING OF A LAYER ARCHITECTURE

a layer is adistributed

software systemoverlay

underlay underlay

members: set Process

overlays: set Layer

locations: Process -> members

sessions: set Flow

underlays: set Layer

attachments: members -> underlays

links: set Flow

routes: set Route

directory of processesin served overlays

exported flows

imported flows

TO UNDERSTAND THE ARCHITECTURE OF LAYERS

overlay

underlay underlay

assumptions

assumptions

requirements

e.g.,once a message is sentby one endpoint of asession, within time tthe layer either deliversit to the other endpointor causes the sessionto fail

e.g.,while a layer processis active in a session,the underlay does not

terminate its attachment

e.g.,while an overlay

process is an endpointof a session, it does

not change locationsin the layer

FOR EACH COMMUNICATION FUNCTION(set of related requirements):

understand the range ofrequirements and assumptions

understand the “design space” ofimplementations, includingapplicability and properties

understand how to map each designinto the layer architecture, retainingseparation of concerns so that otherfunctions will map without interference

understand the principles for allocatinginstances of functions to layers

1

23

4

LAYER




EXAMPLE: MOBILITY

A RESEARCH AGENDA

1

2

3

4

WHAT IS MOBILITY?

A B

a1 a2 b1 b2

WHY MOBILITY FIRST? Because mobility touches on many of themost central concepts of networking, suchas naming and routing, and is very difficultwith the classic Internet architecture.

During the lifetime of aprocess, its registrationin a lower level may change.

Mobility is the communicationfunction that maintains theprocess’s inter-level relationshipsdespite the change.

A B

a1 a2 b

R

ST

r rss

ATTACHMENT MOBILITY

LAYER WITHATTACHMENT

MOBILITY

(INTERNETCORE)

UNDERLAYS

(LOCALAREA

NETWORKS)

a mobile process(here A) is movingphysically, movesbeyond the scopeof current LAN

in the old LAN, itsprocess becomes disconnectedand its attachment is terminated(violating an assumption)

the mobile processregisters witha new LAN

t

this layer must maintain whatit is getting from the underlays, which is the reachability of A

A B

a1 a2 b

R

ST

r rss

ATTACHMENT MOBILITY: THE PRIMARY DESIGN


MOBILITY

(INTERNETCORE)

UNDERLAYS

(LOCAL AREA

NETWORKS)

as A’s attachments change, its linkschange; layer must track these changesand update routes to maintain thereachability of A

t

A’

a1 a2 b

R’

S’

rr

ss

ATTACHMENT MOBILITY: ANOTHER DESIGN


MOBILITY

UNDERLAYS t

A BR

ST

LAYER THATUSED TO HAVEATTACHMENT

MOBILITY

U

UUM

M

M

M

design is good if top layeris very large, becauselarge-scale layer no longerhas to implementmobility

this mobility layer—possibly one of many—has fewer processes anda smaller scope,compared to the top layer

A Bsession

a b1 b2

LOCATION MOBILITY

OVERLAY

(APPLICATIONLAYER)

LAYER WITHLOCATIONMOBILITY

(INTERNETCORE)

a mobile process (here B)changes its location in the layerwhile it is a session endpoint,violating an assumption

this layer must maintain whatit is giving to the overlay, whichis the session between A and B

this might happen becauseB is a process migrating fromserver b1 to server b2

A Bsession

a b1 b2

LOCATION MOBILITY, CONTINUED

OVERLAY

(APPLICATIONLAYER)


(INTERNETCORE)

a mobile process (here B)changes its location in the layerwhile it is a session endpoint,violating an assumption

this might also happen because B’smachine has moved outside the areawhere it can have location-dependentname b1—the process that represents itmust die and be reborn with name b2

x1 x2at the same time, themachine is moving withrespect to LANs, so whyis there no attachmentmobility?

because neither b1 norb2 is mobilein itsattachments!

A Bsession

a b1 b2

when B moves fromb1 to b2, both thelocation directory anda’s session statemust be updated

to:b1to:b2

initially, a and b1cache each other’snames for thissession

LOCATION MOBILITY: THE PRIMARY DESIGN

OVERLAY

(APPLICATIONLAYER)


(INTERNETCORE)

A Bsession

a’ b1’ b2’

LOCATION MOBILITY: ANOTHER DESIGN

OVERLAY


a b1 b2

LAYER THATUSED TO HAVE

LOCATIONMOBILITY

design is good ifbottom layer is verylarge, because large-scale layer no longerhas to implementlocation mobility

this mobility layer—possibly one of many—need serve only oneoverlay

with separation of concerns,implementations of both kindsof mobility compose in thesame layer without interferenceor extra work

the user of thishandheld deviceis engaged in amultiplayer game

gameinstance

during the game session,the user moves

from the range of oneWiFi network to another . . .

. . . and thegame provider

migrates thegame instance

to a lightlyloaded server

COMPOSITION AND SEPARATION OF CONCERNS

S1 S2D

d1 d2

implementation of locationmobility involves only the location directory and session state

implementation ofattachment mobility involves onlyattachments, links,and routing

PARTICULARAPPLICATIONLAYER

INTERNETCORELAYER

LOCALAREANETWORKS

IP MOBILITY IN THE FIELD some Web applications use keysto associate separate sessions

names arelocation-dependent,which is necessary atthis scale

TCP Migrateimplementslocation mobility

[Snoeren &Balakrishnan 00]

Boeing triedimplementingattachmentmobility: ascalingdisaster!

Mobile IPv4, MSM-IP, and i3are all hybrid routingimplementations that giveprocesses multiple ad hocnames, do not satisfy aclean specification

[Perkins 97,Mysore & Bharghavan 97,Stoica et al. 02]

ETHERNETVLAN

implements two-layer design forattachment mobility

**

*

* in design space because they meet all criteria, including usability, composability

although all of thesehave been proposed

for IP mobility,unless we understand

their differences we will not get very far




EXAMPLE: MOBILITY

A RESEARCH AGENDA

1

2

3

4

A RESEARCH AGENDA IN WHICH FIELD?

applications

middleware

transport

network

link

physical

networking

distributedcomputing

TODAY’SCHALLENGES

what I like bestabout Day’s layer definition

is that it organizes networking conceptsso that software experts

can grasp their significance

UNDERSTANDING THEARCHITECTURE OF LAYERS

overlay

underlay underlay


understand requirements, assumptions




12

3

4

LAYER

PROGRESS SO FAR

FOR MOBILITY(at least as understood in thenetworking community):

[Zave & Rexford 11]

UNDERSTANDING THEARCHITECTURE OF LAYERS

overlay

underlay underlay


understand requirements, assumptions




12

3

4

LAYER

WORK TO BE DONE:

EVERYTHING ELSE!

FUNCTIONS RELATED TOMOBILITY

multihoming

anycast

FUNCTIONS THAT SEEMINDEPENDENT OF MOBILITY

above all, security

however, the distinctionbetween “mobile computing”

and “mobile computation”from security-oriented Ambients

seems related toattachment mobility vs.

location mobility!

[Cardelli 11]

WORK TO BE DONE ON SOFTWARE ARCHITECTURE

members: set Process

overlays: set Layer

locations: Process -> members

sessions: set Flow

underlays: set Layer

attachments: members -> underlays

links: set Flow

routes: set Route

THIS PICTURE SHOWS SOME OF THESTATE THAT MUST BE KEPT IN A LAYER

omitting security, resourcemanagement, and no doubt many other necessities

A LAYER IS A DISTRIBUTEDSOFTWARE SYSTEM—WHATIS ITS ARCHITECTURE, ORRANGE THEREOF?

The answer should make itpossible to . . .

. . . meet needs for performance, reliability, and efficiency

. . . implement various functions such as different kinds of mobility and security without interference, i.e., with separation of concerns

. . . support automated generation of customized layer software

REFERENCES

[Day 08] John Day, Patterns in Network Architecture, Prentice Hall, 2008.

[Clark et al. 05] David D. Clark et al., Tussle in cyberspace: Defining tomorrow'sInternet, IEEE/ACM Trans. on Networking, June 2005.

[Handley 06] Mark Handley, Why the Internet only just works, BT TechnologyJournal, July 2006.

[Gettys 11] Jim Gettys, Bufferbloat FAQ, http://gettys.wordpress.com/bufferbloat-faq, 2011.

David D. Clark, The design philosophy of the DARPA Internetprotocols, Proc. SIGCOMM, 1988.

[Clark 88]

[Loo et al. 05] Boon Thau Loo et al., Implementing declarative overlays, Proc.SOSP, 2005.

[Mysore &Bharghavan 97]

J. Mysore and V. Bharghavan, A new multicasting-based architecture for Internet host mobility, Proc. MOBICOM, 1997.

[Cardelli 11] Luca Cardelli, Mobile computational ambients, http://lucacardelli.name/Ambients.html, 2011.

REFERENCES, CONTINUED

[Roscoe 06] Timothy Roscoe, The end of Internet architecture, Proc. Hotnets V,2006.

[Zave 10] Pamela Zave, Internet evolution and the role of softwareengineering, Proc. Symp. on the Future of Software Engineering, 2010.

[Spatscheck 10] Oliver Spatscheck, Cloud computing and my worries about thenetwork that enables it, workshop presentation, 2010.

Lucian Popa et al., HTTP as the narrow waist of the future Internet,Proc. HotNets IX, 2010.

[Popa et al. 10]

[Perkins 97] C. E. Perkins, Mobile IP, IEEE Communications, May 1997.

[Snoeren &Balakrishnan 00]

Alex C. Snoeren and Hari Balakrishnan, An end-to-end approach tohost mobility, Proc. MOBICOM, 2000.

[Stoica et al. 02] I. Stoica et al., Internet indirection infrastructure, Proc. SIGCOMM,2002.

[Zave &Rexford 11]

Pamela Zave and Jennifer Rexford, The design space of networkmobility, 2011.

slides will be posted by 28 June 2011 at www2.research.att.com/~pamela

how software architecture can make an application-friendly internet

Documents