APIs & Underlying ProtocolsLessons · Semantics · Trends

@andreisavu #APICraftSF

Me

Working on Cloudera DirectorPreviously founder of Axemblr.com and Java User Group organizer

More on LinkedIn & @andreisavu

Goal

Share the idea: There is no silver bullet and any implementation must ack network laws

From my experience with:● using multiple cloud provider APIs● using distributed systems (Zookeeper)● writing clients & APIs (jclouds, Director)

A bit of contextOn the role of APIs & Web Evolution

http://techblog.netflix.com/2015/02/a-microscope-on-microservices.html

How we build modern applications?

As hybrids

Start: as monolithic systems that use external APIs (identity, analytics, payments, email etc.)

Evolve: collections of services with cohesive APIs (map to organizational boundaries)

What is the role of an API?

Message passing protocol for State Machines

Often seen as replacing libraries in monolithic systems. Dangerous: doesn’t account for network semantics (reason why RPC failed)

What is the role of an API?

Message passing protocol for State Machines

Better: Writing and reading messages on a synchronous queue (producer / consumer). Matches transport layer semantics.

Fashion-Driven-DevelopmentAre you done rewriting your app in / on X!?

Reinventing the wheel? Often too young to know better?

(simple vs. simplistic approaches)

Doing CORBA again?! (1991)

Good API / Bad APIacknowledge the distributed nature of the app

actual choice of technology is orthogonal

Network is not reliablePartial failures, Timeouts, Partitions,

At-least once message delivery

Latency is not zeroOr predictable. Jitter. Stranglers.

Bandwidth is limitedFanout. Object size. Compression. Capacity

The network is not secureIdentity, Authorization, Encryption, Throttling

Topology does changeEndpoint discovery. Versioning, DNS

Transport cost is not zeroCPU, Contention, Syscalls, GC

There are multiple operatorsA system controlled by multiple entities

Network is heterogeneousNo one size fits all

Your API is as good as your client library

Users writing their own doesn’t really work (especially for static languages)

Not all protocols are equalWide adoption has important network effects

Gravity. Fashion vs. Technical Merits

How did REST-like succeed?Mix: HTTP, Thin Spec, Partial Adherence

Piggyback on: CDNs, Proxies, Tooling etc.

HTTP/1 → HTTP/2

HTTP/2: new framing layer

maintains HTTP paradigmsame URL structure (including TLS)designed for 1:1 conversion at proxyallows for server push

http://daniel.haxx.se/http2/

APIs better suited for:● low cost batch operations● lower latency● compatibility with next gen CDNs● downside: more complex clients

… but the devil is in the details.

What we can hope w/ HTTP/2?

Take aways

There are no silver bulletsThere has to be substance behind fashionRewriting everything is the normAcknowledge transport layer restrictionsTechnology as a mean-to-an-end

Thanks! Questions?asavu@apache.org

@andreisavu