Application design and the end-to-end arguments

David D. Clark

MIT CFP

MIT Communications Futures Program

Bi-annual meeting, May 30-31, 2007

Philadelphia, PA

This talk is about:

The evolution of thinking around the end-to-end arguments (E2EA). A quick review of E2EA…

Design principles for modern applications. What these have to do with each other. What we might learn about distributed

applications and “network services”.

Some readings van Schewick, Architecture & Innovation: The role of the

end-to-end arguments in the original Internet, PhD, TUB Tim Moors, A critical review of “End-to-end arguments in

system design”, 2002 ICC Chen and Jackson, editors, Commentaries on “Active

network and end-to-end design”, !EEE Network, May/June 1998

Kempf and Austein, The rise of the middle and the future of end-to-end, RFC 3724

Reed, The end of the end-to-end argument, 2002 www.reed.com

Blumenthal and Clark, Rethinking the design of the Internet: the end to end arguments vs. the brave new world, ACM ToIT, 2001

Relating E2EA and applications

To a purist, the E2EA do not speak to the design of applications.

The E2EA are concerned with the communications subsystem and “the rest”.

But today’s applications don’t seem very E2E. Does this matter?

Two quotes from the first paper

“In a system that includes communications, one usually draws a modular boundary around the communication subsystem and defines a firm interface between it and the rest of the system.”

“The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system.”

So what is an end-point?

The ends of the transport connection? Often true but not the definition.

The “ends” of the application? Often true but…

The people using the system? “Excuse me, someone dropped a glass. Would

you please say that again?”

Consider this case

A person in St. Louis (middle of US) doing a “careful file transfer” from San Francisco to Boston. Compute checksum in SF and send to StL to save Move file from SF to Bos Compute checksum in Bos and send to StL. Compare in StL.

Now where are the ends? I claim the real end is the user in St. Louis.

There are at least 3 transport connections.

Why is this being careful? Original argument: the communications

subsystem and the disks are unreliable (for lots of good reasons).

The check of correct operation has to be done at a layer that is aware of the desired semantics of the operation. What does it mean to the application to be

“reliable”? But why are the actions of the end-point

reliable?

In search of reliability

The E2EA seem to equate the end-points with points that are reliable and trustworthy, as well as points where the application code runs. Probably sensible, but not fundamental. I tried to sneak in a new word: “trustworthy”.

A case study--email

Email is delivered in stages: From sender to sender’s SMTP server. From sender’s server to receiver’s server. From receiver’s server to receiver via POP or

IMAP. Good reasons for this design.

Asynchronous delivery, delivery to multiple clients, etc.

The shape of email

SenderSMTPserver

PoP/IMAPserver

Receiver

But is it “E2E”? Answer 1: No. there is no overall confirmation

wrapped around the “outside”. Answer 2: Yes, by definition. E2E does not apply

to apps. All of the servers are part of “the rest”. It is E2E if the communications subsystem is E2E.

Answer 3: Yes. We have used reliable, application-aware parts to compensate for unreliable ones. We deem the servers to be trustworthy and reliable. I trust mine, you trust yours…

Answer 4: No. We trust more than we need to.

Is email reliable?

The more basic question. Usually yes. But when it is not…

Case study from Africa. Flakey ISP led users to invent a “whole new idea”. E2E sequence numbers and acks.

Spam checkers eat good mail. How do we compensate?

Summary so far

A possible re-framing of E2E is “trust-to-trust”. Original: “The function in question can completely and

correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system.”

Revised: “The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at points where it can be trusted to perform its job properly.”

Trust--a recognized issue.

“The decision to implement reliable transfer in the transport layer is not justified on the basis of the end-to-end arguments, but rather on the basis of trust”.

-Tim Moors, A critical review of “End-to-end arguments in system design” 2002

Evolution of E2E objectives Correctness--”careful file transfer”. Generality--support many apps. Open--no constraints on user action.

Lessig and Lemley Innovation--lower barriers.

van Schewick Simplicity--fewer bugs and lower cost in core. Minimality--don’t depend on more than

necessary. Optional function--only add, don’t prevent.

Changing our tune

Original: “The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system.”

1998: “ A function or service should be carried out within a network layer only if it is needed by all clients of that layer, and it can be completely implemented in that layer”.

Re-evaluate the network

Is it that the communication subsystem is “unreliable”, or that it is operated by a stake-holder who may not have interests aligned with the end-user?

End-to-end changes from a technique to achieve correct operation to a technique for tussle.

-van Schewick, Architecture and Innovation: the role of the end-to-end arguments in the original internet, has an excellent analysis of this:

Application design

The original argument--the communications subsystem and the rest--seems to be specific to the network layer.

A discussion of trust and alignment of interests is not restricted to any layer.

So perhaps this broader interpretation can help us understand how to design applications.

First fact of life today

Applications are not simple, two-party transfers. They are “full of “ servers and services.

Email Web: caches, proxies, Akamai, etc. IM, games, etc.

Begs the question--is the design compatible with actual trust assumptions? T2T would argue: put the function where “I” trust it will

be carried out properly.

Second fact of life today

We don’t trust each other. Contrast with original “careful file transfer”.

Each party may delegate to servers that it trusts.

The shape of email

SenderSMTPserver

PoP/IMAPserver

Receiver

My sphereof trust

Your sphereof trust

Any trust here?

Two examples Email: protocols allow functions to be

assigned to different servers. Can re-arrange to match trust (and other)

assumptions. The story from Africa: ISP “forced” users to

use their mail server. IM: the most popular design requires

users to trust (use) a third-party server, whether or not they want to. Sort of…

If we equate E2E with trust In particular, the assignment of function to nodes

that are trusted to carry them out . Then third party servers and services are not

intrinsically in violation of E2E. Trust--depends on choice. Minimality--why trust them? Open--can anyone deploy them? Generality--does not seem to be an issue. Simplicity--keep the core simple. Is this

relevant?

Third fact of life today

We cannot trust our own end-computers. Perhaps the final insult to classic E2E.

And we may not want to maintain them. So trust may imply movement away from

end-computers on to specialized servers. Simplicity may imply similar movement.

Back to lack of mutual trust

What do we do when we don’t trust each other but want to interact? A real life question.

1) Put up lots of defenses and proceed with caution.

2) Utilize trusted third parties. Real life is full of these--credit card companies,

registries of deeds.

Some classic E2E arguments (minimality, trust the edges) might imply a preference for 1. This make sense?

The shape of trust in email

SenderSMTPserver

PoP/IMAPserver

Receiver

My sphereof trust

Your sphereof trust

Reasoning by visual analogy: two “distributed ends”.Sort of E2E-like…

Third parties

Customer

Creditcard co.

Merchant

My sphereof trust

Your sphereof trust

This is “trust in the middle”, not the edges.

Reasoning by analogy The E2EA arguments focus on the end in the

context of a communications subsystem that is: Perhaps untrustworthy. “Naturally” unreliable.

These factors do not generalize. Services in the middle can be picked based on

determination of trust. They are not “naturally” unreliable.

The residual argument is minimality. There may be a high price for being unwilling to trust

a third party.

Yes, there is a relevant E2EA

It is about moving function to places where it can be trusted to be carried out.

It implies choice by the end-users. End-user choice empowers the user.

Trust as the user pleases. Select applications, servers and services as the

user pleases. The goals of generality, innovation, lack of

constraint are served by choice.

The unstated “other half”

The E2EA presume that you can, if you choose, build a general, application-independent useful service. Like packets. But the discovery of packets is not a

consequence of the E2EA. It is the success of packets that make the E2EA relevant.

How does this relate to applications?

Generalities at the application

Open platforms for services and servers. Like PlanetLab.

Services that cross applications. Identity management Trusted agents. Location and presence management.

Does any form of the E2EA speak to the design of these?

Design options

Allow for options and alternatives. Different actors offering the “same” service. Different forms of the service for different

contexts. Let the end-user have choice and pick.

Each end may want to pick independently.

Generalizing the E2EA…

In 3 easy steps. E2E -> T2T.

Position function where it can be trusted to do its job. Position trusted function around untrusted/unreliable

components to compensate and correct for them. There can be more than 2 points of trust.

T2T2T, etc. Balance minimality with utility.

The actual end-points (computers) may not be fully trustworthy. The user (person) at the end is the ultimate locus of trust. This trust is expressed through choice, among other ways.