ljc mashup "building java microservices for the cloud && chuck norris doesn't need...

Building Java (micro)services for the CloudThe DHARMA principles

Daniel BryantPrincipal Consultant, Open Credo

[email protected]@danielbryantuk

&&

Chuck Norris Doesn’t Need DevOps…but Java Developers might benefit

Who Am I?

• LJC Associate

• Adopt OpenJDK and JSR

10/11/2014 @danielbryantuk

• Principal Consultant at OpenCredo

Agile transformations

DevOps, Cloud and Microservices

Big Data

The Current Industry Wish List…

• Service-Oriented Architecture

• Cloud-based deployments

• DevOps Culture

@danielbryantuk10/11/2014

The Current Industry Wish List…

• Service-Oriented Architecture– muCon 2014 (coming soon via skillsmatter.com)

• Cloud-based deployments– JavaOne 2014 (coming soon via parleys.com)

• DevOps Culture– Devoxx UK talk (bit.ly/1BylnZb)

– Previous LJC Event (bit.ly/1elVPJz)


http://bit.ly/1BylnZb

http://bit.ly/1elVPJz

So, what’s all this I hear about small services?…


“Microservices”aka

• “SOA done right”

• “SRP” services– “Java, The Unix Way” (bit.ly/1cX8VsS)

• “Small” codebase services – 1000 LOC… 100… 10…?

• “12 Factor apps” (12factor.net)

• My personal opinion…


http://bit.ly/1cX8VsS

http://12factor.net/

“Can I fit the service in my head?”


Moving to the Cloud – common problems…



Not respecting the underlying environment


Lack of application/platform monitoring…


Bizarre failure modes…


Difficulty in understandingthe new architecture


Not testing in the Cloud…(hint: here be dragons!)

We’ve created the “Cloud DHARMA Principles” to act as a checklist when building Cloud apps


Documented (just enough)

Highly cohesive/loosely coupled (all the way down)

Automated from commit to Cloud

Resource aware

Monitored thoroughly

Antifragile


API Docs with Swagger


helloreverb.com/developers/swagger

https://helloreverb.com/developers/swagger

Create a PACT


github.com/DiUS/pact-jvm

https://github.com/DiUS/pact-jvm




Resource aware


Antifragile


High Cohesion / Loose Coupling(all the way down…)

• Code

• Architecture– Components

– Services

• Public API– PayPal (bit.ly/1hnZNly)


http://bit.ly/1hnZNly

Microservices: Architectural Silver Bullet?

• Enforce service boundaries– Bounded contexts

– Domain-Driven Design

• Separation of Concerns– Encapsulate what varies

– Easier to scale/tune independently

• Is this a free-lunch? (bit.ly/1gSw4L7)


http://bit.ly/1gSw4L7

No!! It’s all too easy to…

1) Create a distributed monolith

2) Push complexity into orchestration


www.codingthearchitecture.com/2014/07/06/distributed_big_balls_of_mud.html

http://www.codingthearchitecture.com/2014/07/06/distributed_big_balls_of_mud.html

Sneak-peak: Smashing the Monolith…

• Business functionality -“Cart Service”– Noun, verb, SRP (slidesha.re/1owdJhh)

• Technology chunk - “Email Service”

• Vertical Slice - “Service per page” – Groupon (vimeo.com/105880150)

• Horizontal Slice - “User Repo” – An anti-pattern?


http://slidesha.re/1owdJhh

https://vimeo.com/105880150

DZone’s Enterprise Integration Guide


www.dzone.com/page/research

http://www.dzone.com/page/research




Resource aware


Antifragile


Thou Shalt Know thy Cloud…

“Everything fails all the time [in the cloud]”

Werner Vogels, CTO, Amazon.com

• Everything is ephemeral

• Volatility

• Noisy (virtual) neighbours– bit.ly/1w1HQy7


http://bit.ly/1w1HQy7




Resource aware


Antifragile


Antifragile

• The opposite of fragile?

– Robust…

– Antifragile…

• Netflix are best-in-class

– bit.ly/1gs5n3q

• System must be robust first!


http://bit.ly/1gs5n3q

Design for Failure

• Distributed Computing Principles

– ‘For young bloods’ (bit.ly/1pKVepz)

– Eight Fallacies… (bit.ly/1zCBet7)

• Design patterns

– Timeouts / retries

– Bulkheads / circuit-breakers


http://bit.ly/1pKVepz

http://bit.ly/1zCBet7

Retries


github.com/rholder/guava-retrying

https://github.com/rholder/guava-retrying


github.com/Netflix/Hystrixprojects.spring.io/spring-cloud/

Circuit-breaker/bulkhead

https://github.com/Netflix/Hystrix

http://projects.spring.io/spring-cloud/

Antifragile Patterns: Elastic Scaling

Stateless components

Distributed data stores / caches





Resource aware


Antifragile


Microservices in the Cloud…

and now to DevOps…

…or as the recruiters like to say

“DeeVoips”, “DevAnOps” and “Dev-Oops”


What’s in a Name?

• “Development / Operations”

• Increasing cohesion between:– Business

– Development

– Quality Assurance

– Operations

“Bu-Dev-Qa-Ops”?


So, What’s Chuck Norris Doing Here?


Chuck Norris doesn’t need DevOps…

…as a one-man army he codes with one hand, tests with the other and deploys with his beard


The rest of us…

…work in teams to develop software

Software Development Teams

• Developers

• Quality Assurance

• Operations



• Developers


• Operations


This silo mentality has to stop


Culture is Vital

• Culture drives behaviour, drives culture…

– “Communication, simplicity, feedback, courage”

– Everyone is responsible for delivery

– Continuous experimentation and learning

• Not easy to change culture

– The hardest part of DevOps…

– …but you will learn new things


Changing Culture

• Create an effective team…

• “Habits of highly effective technical teams”

– Martijn Verburg (bit.ly/1aF9SnK)

• “Patterns of Effective Teams”

– Dan North (vimeo.com/68226771)


http://bit.ly/1aF9SnK

http://vimeo.com/68226771

Chuck Norris doesn’t do QA…

…Chuck Norris can test an entire application with a single assert

(and get 110% code coverage)


The rest of us…

…need high-quality automated QA

Automating QA

• Intra-component integration testing

– Utilise embedded datastore/middleware

– Cucumber (via ports/adapters/API?)

• Fault-tolerance

– Chris Batey’s Skillscast (bit.ly/1tU6wZj)

– WireMock + Saboteur (wiremock.org)

– “Scassandra” (github.com/scassandra)


http://bit.ly/1tU6wZj

http://wiremock.org/

https://github.com/scassandra

Automating QA

• Inter-component integration testing

– The hardest part of SOA…

– Consider ‘synthetic txns’ (active monitoring)

• Service virtualisation

– Mountebank (www.mbtest.org)

– Mock external services (e.g. Spring profiles)


http://www.mbtest.org

All arrays Chuck Norris creates are of infinite size…

…as Chuck Norris knows no bounds


The rest of us…

…should manage our resources and cultivate ‘mechanical sympathy’

Cultivating “Mechanical Sympathy”

• Virtualisation– Tech Target (bit.ly/1kDVqyG)

• Networking– ‘Unix and Linux System Administration Handbook’

– AWS docs aws.amazon.com/documentation

• Thinking/Acting Operationally– You write it, you run it… (“dev on call”)


http://bit.ly/1kDVqyG

http://aws.amazon.com/documentation/

When Chuck Norris throws Exceptions…

…everybody knows about it because they land outside of the data center


The rest of us…

…should log all errors (and other vital information for diagnostic purposes)

Monitor All The Things!

• Infrastructure monitoring– Nagios / Zabbix

– Codahale, Actuator

• Distributed Tracing – twitter.github.io/zipkin

• Centralised Logging– logstash.net


http://twitter.github.io/zipkin/architecture.html

http://logstash.net/

The ‘ELK’ Stack


blog.comperiosearch.com/blog/2014/08/14/elk-one-vagrant-box

http://blog.comperiosearch.com/blog/2014/08/14/elk-one-vagrant-box/

“…but what should I graph?”



Phrase borrowed from Etsy!

Thanks For Listening

• Massive thanks

– OpenCredo (@OpenCredo)

– notonthehighstreet.com

– RecWorks & The LJC

• Questions / comments?

– [email protected]

– @danielbryantuk


mailto:[email protected]

http://www.twitter.com/danielbryantuk