M E S OS F RA M E W OR K AP I V1

@vi n od ko n e

M E SO S A P I S

S che d u l e r M as t e r Ag e n t Exe cu to r

Scheduler API Executor APIInternal API

Operator API

Framework API

N E T W O R K I N G C O N ST RA I N T S

S che d u l e r M a s te r

Firewall

Containers :( Network partitions :(

DE P E NDE NC E O N NAT IVE L IB RA RY

Sch e d u l e r

M a s te rJ a v a L i b r a r y

Na t i v e L i b r a ry

mesos.jar libmesos.so Hard to debugNot portable

Un-needed dependencies

UND O CU M E NT ED F RAM EW OR K AP I

POST /master/mesos.internal.LaunchTasksMessage HTTP/1.1

User-Agent: libprocess/scheduler-1234-23-23342@127.0.0.1:8081

Libprocess-From: scheduler-1234-23-23342@127.0.0.1:8081

Connection: Keep-Alive

Host: Transfer-Encoding: chunked

LIBPROCESS_IP

LIBPROCESS_ADVERTISE_IP

L AC K OF A P I V E R S I O N I N G

S ch e du l e r M a s te r

Version YDriver version XCompatible?

H A R D T O E VO LV E TH E A P I

Lot of boiler plate to expose new API objects

Adding new features breaks the APIMaintenance API?

– HA RVE Y DEN T

“You either die a hero…or live long enough to see yourself become the

villain” replaced by a better API

F RA M E W O R K A P I V 1 G OA L S

API versioning

Easy to evolve API

Well documented

Allow pure language client libraries

Work inside firewalls / containers

F RA M E W O R K A P I V 1

• Calls and Events as building blocks

• Schema: v1/scheduler.proto and v1/executor.proto

• Versioned API endpoints

• master: /api/v1/scheduler

• agent: /api/v1/executor

CAL L S Old APISU BS CR I B E start()TEAR DOWN stop()

ACCE PT acceptOffers()DE CL I NE declineOffer()R EV I VE reviveOffers()

K I LL killTask()SH U TDOWN * Shutdown executor *

ACK NOWL EDG E acknowledgeStatusUpdate()

R ECONC I LE reconcileTasks()MESS AG E sendFrameworkMessage(

)R EQU ES T requestResources()SU P P R ES S suppressOffers()

EVENT S Old API

SU BSC R I B ED registered() / reregistered()

OF FE R S resourceOffers()R ES C I ND offerRescinded()U P DATE statusUpdate()

MESS AG E frameworkMessage()

FA I LU R E slaveLost() / executorLost()

ER R OR error()H EARTB E AT * Periodic heartbeats *

PR O T O C O L

• HTTP 1.1

• Scheduler/Executor opens connections to the master/agent

• A persistent connection to receive Events

• One (or more) connection(s) to send Calls

PR O T O C O L

• Every call is a HTTP POST request

• Content-Type: application/json or application/x-protobuf

• SUBSCRIBE call results in a “200 OK” streaming response

• Record-IO formatted events

• “Mesos-Stream-Id” header

• All successful non-SUBSCRIBE calls result in “202 Accepted”

• Must include “Mesos-Stream-Id" header

SU BS C R I PT I O N R E Q U E ST

POST /api/v1/scheduler HTTP/1.1

Host: masterhost:5050Content-Type: application/jsonAccept: application/jsonConnection: close

{ “type” : “SUBSCRIBE”,

“subscribe” : { “framework_info” : { “user” : “foo”, “name” : “Example HTTP Framework” }, }}

SU BS C R I PT I O N R E S PO N SE

HTTP/1.1 200 OK

Content-Type: application/jsonTransfer-Encoding: chunkedMesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af

<event length>{ “type” : “SUBSCRIBED”, “subscribed” : { “framework_id” : {“value”:“12220-3440-12532-2345”}, “heartbeat_interval_seconds” : 15 } }<more events>

K I L LPOST /api/v1/scheduler HTTP/1.1

Host: masterhost:5050Content-Type: application/sonAccept: application/jsonConnection: closeMesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af

{ “framework_id” : {“value” : “12220-3440-12532-2345”},

“type” : “KILL”,

“kill” : { “task_id” : {“value” : “12220-3440-12532-my-task”}, “agent_id” : {“value” : “12220-3440-12532-S123345”} }}

Response: HTTP/1.1 202 Accepted

DI S C O N N E C T I O N S & PA RT I T I O N S• Master tracks the persistent subscription

connection

• Scheduler can reconnect within failover timeout

• Assigns stream IDs to uniquely identify streams

• Periodic HEARTBEATs sent by master

• Scheduler can reconnect if no HEARTBEATs

V E R S I ON I N G

Explicit Simple

Avoid version explosion

/api/v1/scheduler/api/v1/executor

AP I VE R S I O N VS R E L E AS E VER S I O N• API version == Major release version

• v1 API supported by 1.0.0, 1.4.0, 1.20.0

• vN API released in N-1 release version

• vN API considered stable in the last N-1 release

• Version bumping

• Major/API version bumped for backwards incompatible changes (> yearly)

• Minor version bumped regularly (4-8 weeks)

SC H E D U L E R A P I V 1 STATU S• 0.28.0: More API changes (Use this version!)

• 0.24.0: MVP

• Experimental: Use in testing clusters only

• Schema: v1/scheduler/scheduler.proto

• Documentation: http://mesos.apache.org/documentation/latest/scheduler-http-api/

• Reference implementation

• C++ library: https://github.com/apache/mesos/blob/master/src/scheduler/scheduler.cpp

• C++ scheduler: https://github.com/apache/mesos/blob/master/src/examples/test_http_framework.cpp

E XE C U T O R A P I V 1 S TATU S

• 0.28.0: MVP

• Experimental: Use in testing clusters only

• Schema: v1/executor/executor.proto

• Documentation: http://mesos.apache.org/documentation/latest/executor-http-api/

• Reference implementation

• C++ library: https://github.com/apache/mesos/blob/master/src/executor/executor.cpp

• C++ executor: https://github.com/apache/mesos/blob/master/src/examples/test_http_executor.cpp

TO D O

• Authentication support

• Encryption support

• Rate limiting support

• Scheduler <—> Executor message optimization

T H A N K YO U

Questions?