l11 service design and rest

HÖNNUN OG SMÍÐI HUGBÚNAÐAR 2015L11 SERVICE DESIGN AND REST

Agenda

How to Organise SystemsAPIs and ServiceService ArchitectureHow to Model ServicesREST

Reading

Microservices Grein eftir Martin Fowler og James LewisMartin Fowler - Microservices Youtube video, Martin Fowler

REST+JSON API Design - Best Practices for Developers Youtube video, Les Hazlewood, CTO of Stormpath

How to Organise Systems

Conwey’s Law

Organisations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations

Team A Team B

Communication path:Team B needs services from team A

API, documentation, and shared model

Conwey’s Law

Priorities of teams are not shared

Team Communications with other teamsTwo ways:

Eliminate the communication path, possibly by changing the team structure

Recognise that there is a communication path and establish way for people to work successfully

Remember Conway

Responsibility of Teams

Teams must have complete responsibility and ownership of their services

Must have all the required skill set to avoid dependence on

Service architecture is very much a human thing

APIs and Services

Trends in Architecture

Service Oriented Architecture dates back to mid 1990s

Web Services meaning XML and SOAP using an Enterprise Services Bus

Confusions on terminology

Trends in Architecture

SOA as in Web Services using SOAP

1. All teams will henceforth expose their data and functionality through service interfaces

2. Teams must communicate with each other through these interfaces3. There will be no other form of interprocess communication allowed4. It doesn't matter what technology they use5. All service interfaces, without exception, must be designed from the

ground up to be externalizable. No exceptions.6. Anyone who doesn't do this will be fired.

Bezos’ Mandate (from Yegge’s Rant)

Service Oriented Architecture

SOA actually means that components of an application act as interoperable services, and can be used independently and recombined into other applications.

Engineering Software as a Service by David Patterson and Armando Fox

Microservices

In recent years a new term has emerged, Microservices:

The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.

From the Lewis & Fowler article

Martin Fowler - Microservices https://www.youtube.com/watch?v=2yko4TbC8cI

Martin Fowler Microservices

https://www.youtube.com/watch?v=2yko4TbC8cI

Definition of SOA is useless because it is so overloadedMicroservice architecture is better but needs to be clarified

Service Architecture

Each company should define what they mean by Service Architecture

The Right Way


A component is a software building block that is

independently replaceable

independently upgradable

Component


Service Architecture means that components of an application act as

interoperable services,

and can be used

independently and recombined into other applications

Service is a component that the replaceable and independently upgradable

Definition of a Service

SERVICE

Implementation

Service API

Service uses other service

Services are like the Unix commands where output from one is in input to another

SERVICE

Implementation

ServiceAPI

SERVICE

ImplementationJson

Definition of a Service

Single Responsibility Principle

Gather together those things that change for the same reason, and separate those things that change for different reasons

Service boundaries focus on business boundaries

Separate Variations Principle

Identify the aspects of your application that vary and separate them from what stays the same

Service vs. Monolith

Service is independently deployable

Monolith applicationSingle Deployment Unit

Service applicationMultiple Deployment Unit

Scalability is provided with multiple machines - Y scalingPossibility of better load management

Service vs. Monolith

Monoliths deployed on multiple machines

X Scaling

Multiple Services deployed on multiple machines

Y Scaling

In Service Architecture we define services as a components that communicate with a REST based HTTP remote call and AMQP message

Componentization via Services

A service is a replaceable unit, it is independently replaceable and upgradeable

A service has a published interface - an APIThe only way to use the service is to use the APIEverything else in the service is encapsulated, including the data

REST

AMQP

Level 2 or 3 with JSon content

Message Protocol

Service API

This is how teams communicate

Feature teams provide much better knowledge in the domain

Team owns the entire service from UI to DB

Organised Around Business Capabilities

In monolith architecture it makes sense to have front-end developers as a team and business logic people as another

Monoliths can Organise Around Layers


Due to Conway’s Law this organisation leads to bad systems, with lots of duplication of effort


The only thing that is worse is to allow developers of one team to inherit classes that are the responsibility of another team

Formula for disaster!

Organised Around Business Capabilities

Communication paths should as few as possible - teams should be fairly independent

Any problematic communication path needs to be address

Software can be developed as projects, usually for others

Software can be developed as products for B2C or B2B markets

When building a product, a team is responsibility the whole production of the product. Writing it, testing it, deploying it, running it.

Companies need to figure out the right way

Products not Projects

The traditional SOA style applications favours using specific routing system to facilitate service communication, for example an Enterprise Service Bus (ESB)

Service Architecture favours the microservice approach: smart endpoints and dumb pipes

This means that services use REST API to communicate and each service is a dedicated business logic and output in form of JSon

Smart endpoints and dumb pipes

Smart endpoints and dumb pipes

System A

System B

System C

Enterprise Service BusRouting Logic

Routing Logic

Routing Logic

System A

System B

System C

Routing Logic

Routing Logic

Routing Logic

Network

Each service can be done in different programming language and with different database

While such polyglot environments are more challenging to manage, the possibility of using the best technology for the given problem has true benefits

The goal is to make teams more responsible for the code they write and test

Teams are responsible for coding, testing and deploying their own services

Decentralised Governance

Data management is part of servicePossibilities of differentdata storesPossibilities of data scaling

Decentralised Data Management

With multiple services, each deployable, the complexity of managing the software deployment and monitoring becomes harder

The process of building, deploying and operating services needs further developed to meet continuous delivery

One thing that must be looked into is to automate more tasks

Infrastructure Automation

Services will depend on other services

How to design if a dependant service is not available?

We must look into fall-back routines and other mechanisms to build a robust systems

Use latency and fault tolerance support

Design for Failure

How to Model Services

Term microservices is sometimes used, but is misleadingHas nothing to do with lines of code

How big is a service?

Example definition:

Balance between integration points and size

Time: Can be rewritten in one iteration (2 weeks)Features: All things that belong together

Loose CouplingWhen services are loosely coupled, a change in one service should not require a change in another

A loosely coupled service knows as little about the services with which it collaborates

Source: Building Microservices

High CohesionWe want related behaviour to sit together, and unrelated to sit elsewhere

Group together stuff the belongs together, as in SRP

If you want to change something, it should change in one place, as in DRY


Bounded ContextConcept that comes from Domain-driven Design (DDD)

Any given domain contains multiple bounded contexts, and within each are “models” or “things” (or “objects”)

that do not need to be communicated outside

that are shared with other bounded contexts

The shared objects are define the explicit interface to the bounded context


Bounded Context

Source: Martin Fowler, BoundedContext http://martinfowler.com/bliki/BoundedContext.html

http://martinfowler.com/bliki/BoundedContext.html

Bounded ContextConcept that comes from Domain-driven Design (DDD)

Any given domain contains multiple bounded contexts, and within each are

objects that do not need to be communicated outside

objects that are shared with other bounded contexts

The shared objects are define the explicit interface to the bounded context

Source: Building Microservices (figure is from the book)

What is REST?▪ An architectural style– Representational State Transfer▪ REST is defined in Roy Fielding’s dissertation from 2000

Adoption of REST

Why REST▪ Scalability▪ Generality▪ Independence▪ Latency▪ Security▪ Encapsulation

Les Hazlewood, CTO of Stormpath REST+JSON API Design - Best Practices for Developers

HATEOASHypermedia As The Engine Of Application State

What is REST?

▪ Quick and insufficient explanations:– REST is nothing more than Web Service with the fancy URIs– REST = CRUD – that is,

– POST, GET, PUT, DELETE = Create, Read, Update, Delete

– Almost but not quite

Better understanding▪ REST stands for Representational State Transfer it is an

Architectural Style for distributed hypermedia systems– It has nothing to do with XML, Json, HTTP or the Web– Jason is one way to represent resources, but not the only

way

Better understanding▪ HTTP/1.1 was built in a RESTful manner– The Web is the largest known implementation of a system

conforming to the REST architectural style▪ REST is not CRUD– There are some similarities, but ‘similar’ is far from ‘same’– In best case CRUD is a subset of REST

Better understanding▪ Representational State Transfer

Client gets information about the data from the SERVER

CLIENT SERVER

RESOURCE DATASESSION DATA

URI REQUEST

RESOURCE

HYPERMEDIA Stateless

Uniform Interface▪ REST is defined by 4

interface constraints: 1. Identification of resources2. Manipulation of resources

through representations3. Self-descriptive messages4. Hypermedia as the engine of

application state (HATEOAS)

Richardson Maturity Model (RMM)

▪ Level 0– SOAP, XML RPC, POX– Single URI▪ Level 1

– URI Tunnelling– Many URI, Single verb

▪ Level 2– Many URIs, Many verbs– CRUD services ▪ Level 3

– Level 2 + Hypermedia– RESTful Services

Level 0▪ Everything is packed in XML– Ignoring all HTTP features– SOAP envelope– Ignores all status codes – 55 codes (including “418 I’m a

tepot”)

Level 1▪ We understand that if every resource can be identified by a

URI, then we can POST to that URI for some expected result– Many URI, one request method– Still not good enough, as we skip some of the benefits of

the protocol– POST cannot be cached

Level 2

▪ Different verbs or request methods (GET, PUT, and DELETE) – and use them, as they were intended

Safe IdempotentGET Yes YesPUT No YesDELETE No YesPOST No No

Level 3▪ HATEOAS – Hypermedia as the engine of Application State– Don’t hardcode your values, use the navigational aspect of

the web– Hardcoding is easy and fast, but hard to change – in a

distributed world, hard becomes ‘nearly impossible’– With POJOs we don’t hardcode the memory address, we

use pointers

Level 3

▪ Why use the values of the second column if we can agree on the keys for these value – the actions?

Action URIAdd new comment to a blog POST /blog/17/comment/Get all the comments for blog entry number 17

GET /blog/17/comment/

Get user 167671 GET /user/167671Update a user number 167671 PUT /user/167671

Level 3

▪ The response of “get user 167671” will contain keys like “UPDATE” and the value is “http://example.is/user/167671”

Action URIAdd new comment to a blog POST /blog/17/comment/Get all the comments for blog entry number 17

GET /blog/17/comment/

Get user 167671 GET /user/167671Update a user number 167671 PUT /user/167671

http://example.is/user/167671

Level 3{ "id": "98423808305", "from": { "name": "Coca-Cola", "category": "Consumer_products”, "id": "40796308305" }, "name": "A spectacular artwork made solely from used aluminum cans has been unveiled on top of the chalk cliffs of the Sussex coastline to mark the beginning of Recycle Week.", "picture": "http://photos-e.ak.fbcdn.net/hphotos-ak-snc1/hs085.snc1/5041_98423808305_40796308305_1960517_6704612_s.jpg", "source": "http://sphotos.ak.fbcdn.net/hphotos-ak-snc1/hs085.snc1/5041_98423808305_40796308305_1960517_6704612_n.jpg", "link": "http://www.facebook.com/photo.php?pid=1960517&id=40796308305", "comments": { "data": [ { "id": "98423808305_1152797", "from": { "name": "Caitlin Catherine Kennedy", "id": "1658825260" }, "paging": { "next": "https://graph.facebook.com/98423808305/comments?limit=25&offset=25" } }}

HATEOAS

Resources▪ Nouns, not verbs– Examples: Account, Group, Customer– Not behaviour▪ Two types– Collection of resources– Instance of a resource

/applications /applications/7828

REST Behaviour▪ GET to get resource▪ POST to add resource▪ PUT to update resource▪ DELETE to delete resource▪ HEAD to get header information

GET someservice.com/api/customer/3829

POST someservice.com/api/customer/

REST examples

PUT someservice.com/api/customer/3829

Getting

Adding

Updating

GET▪ On the parent resource: collection

GET /users

200 OK { “users”: [ { “username”: “olandri”, ... }, { “username”: …

}

▪ On the parent resource, give me a specific instanceGET /users/789

200 OK { { “username”: “olandri”, ... } }

GET

POST as Create▪ On the parent resource

POST /users { “username”: “olandri”, “firstname”: “Ólafur Andri”, “lastname”: “Ragnarsson” }

201 Created Location: https://api.rufan.com/users/789

PUT as Update▪ On the parent resource

– Allows partial updates, which is important for complex data type

PUT /users/789 { “username”: “olandri”, }

200 OK

Endpoint considerations▪ For internal use, you can use anything– http://www.foo.com/dev/api/

▪ But if you want others to use– http://api.rufan.com/v1.0/

http://api.rufan.com/v1.0/

Versions▪ Two ways

▪ URL– https://api.rufan.com/v1

▪ Media-Type– Application/json;application&v=1

JSON ▪ JS in JSON is JavaScript▪ XML is also possible▪ JSON is fits well with JavaScript manipulation▪ Libraries make all this easier▪ Maps well to objects like POJOs▪ Compact format, much better space complexity than XML

Example{ "firstName": "John", "lastName": "Smith", "isAlive": true, "age": 25, "height_cm": 167.6, "address": { "streetAddress": "21 2nd Street", "city": "New York", "state": "NY", "postalCode": "10021-3100" }, "phoneNumbers": [ { "type": "home", "number": "212 555-1234" }, { "type": "office", "number": "646 555-4567" } ], "children": [], "spouse": null }

Some design rules:

Determine the resources firstTransparant as opposed to opaqueLowercaseAs Java package, domain specific with general terms closer to the rootDesign by intention of resultsResource names are plural, customers not customerIf /a/b is valid, so is /a

Endpoint Design

/customers/{id}

/customers/117 Returns customer with id=117

Use { } for parameters

▪ Example: School, Student, and Classes in their normal relationship– schools have zero or more students– schools have zero or more classes– students have zero or more classes– classes have zero or more students

Resource Oriented Design


http://example.com/lms/schools => list of schools http://example.com/lms/schools/{school} => info about one school http://example.com/lms/schools/{school}/students => list of students http://example.com/lms/schools/{school}/students/{student} => info on one student http://example.com/lms/schools/{school}/students/{student}/courses => list of courses (as links, not full resources) student is enrolled in http://example.com/lms/schools/{school}/courses => list of courses http://example.com/lms/schools/{school}/courses/{course} => info on one course http://example.com/lms/schools/{school}/courses/{course}/students => list of students (as links, not full resources) enrolled in course

http://www.example.com/lms/students/123

{ "ID" : 123, "name": "Name of Student", "SSN" : 123456789, "DateOfBirth" : "2001-01-01" }


http://www.example.com√lms/students/123

{"Classes" : [ { "name" : ”History", "link" : " http://www.example.com/classes/117" }, { "name" : ”New Technology", "link" : " http://www.example.com/classes/118" } ]}


Architecture

Client

DomainData

Source

CMS

RESTWeb

Server DB

JavaScript calls REST API to get

Json array

What REST is not▪ Silver bullet▪ Not easy although simple▪ Isn’t tied to the web▪ Not perfect– Client holds application state – Latency– Integrity

Summary▪ REST is HTTP done right▪ Architecture style▪ Not so easy to get right▪ Trend is towards APIs