unlocked workshop oscon 2013 - part i

OSCON

23 july 2013

Portland, ORButter Up your application

Tuesday, July 23, 13

welcome & introductions

Tuesday, July 23, 13

your unlocked team

3

Wayne A. Walls@waynewalls

Alex Brandt@alunduil

Hart Hoover@hhoover

Tuesday, July 23, 13

get social!

4

@rackspace #unlockedPDX#oscon

Tuesday, July 23, 13

5

Tuesday, July 23, 13

6

Business Strategy

Tuesday, July 23, 13

7

Cloud Savvy

Tuesday, July 23, 13

8

Cloud Freedom

Tuesday, July 23, 13

9

Application Fit

Tuesday, July 23, 13

cost of cloud apps

Wayne wallscloud evangelist

Tuesday, July 23, 13

11

Tuesday, July 23, 13

40%PRIVATE CLOUD

11

Tuesday, July 23, 13

40%PRIVATE CLOUD

11

50%HOSTED ON-SITE

Tuesday, July 23, 13

90% of Workloads Remain In-House...

40%PRIVATE CLOUD

11

50%HOSTED ON-SITE

Tuesday, July 23, 13

Cost Considerations

Infrastructure

12

Tuesday, July 23, 13

13

How many customers can you serve?

Tuesday, July 23, 13

14

How easy would it be to scale?

Tuesday, July 23, 13

Innovation

15

Tuesday, July 23, 13

In-house?

16

Tuesday, July 23, 13

Cost Considerations

17

Developers

Tuesday, July 23, 13

18

Developer time translates into $

Tuesday, July 23, 13

19

Apply effort where it matters

Tuesday, July 23, 13

20

Be realistic about engineering time

Tuesday, July 23, 13

21

Choose tools that scale

Tuesday, July 23, 13

everything as code

Wayne wallscloud evangelist

hart hoovercloud evangelist

Tuesday, July 23, 13

Tuesday, July 23, 13

24

Self-service

Tuesday, July 23, 13

25

On-demand

Tuesday, July 23, 13

26

Metered

Tuesday, July 23, 13

27

Resource pooling

Tuesday, July 23, 13

28

Broad network access

Tuesday, July 23, 13

NIST SAys...

29

Tuesday, July 23, 13

NIST says...

30

Tuesday, July 23, 13

NIST says...

31

Tuesday, July 23, 13

multi-cloud

32

Tuesday, July 23, 13

ENTER devops

33

Tuesday, July 23, 13

34

Culture & Work Methodology

Tuesday, July 23, 13

35

Tooling

Tuesday, July 23, 13

36

Deployment Strategies

Tuesday, July 23, 13

37

Measure All the Things

Tuesday, July 23, 13

38

Automate All the Things

Tuesday, July 23, 13

39

Tuesday, July 23, 13

40

Tuesday, July 23, 13

41

Gerrit

Tuesday, July 23, 13

42

Gerrit

Tuesday, July 23, 13

43

Tuesday, July 23, 13

44

Tuesday, July 23, 13

Infrastructure as code

45

Tuesday, July 23, 13

46

Same Problems

Tuesday, July 23, 13

47

What version is in production?

Tuesday, July 23, 13

48

Issues are found, fixed & redeployed quickly

Tuesday, July 23, 13

49

Test as you develop

Tuesday, July 23, 13

50

Same Best Practices

Tuesday, July 23, 13

51

Tag, branch & release code that defines infra

Tuesday, July 23, 13

52

Coverage lifecycle over various stages

Tuesday, July 23, 13

53

Continuously Test

Tuesday, July 23, 13

54

Tuesday, July 23, 13

Chef Server

Chef Workstation

Chef Client

knife

node

Chef Client

node

environment

55

Tuesday, July 23, 13

•Nodes: Servers

•Environments: Groups of nodes (Dev/QA/Prod)

•Recipes: Describes the desired state of a node

•Cookbooks: Group of Recipes

•Roles: Group of Cookbooks (Web)

•Attribute: information about the current state of a node

56

Tuesday, July 23, 13

Thinking like a developer57

Tuesday, July 23, 13

Product Backlog

Sprint Backlog Sprint Working increment

of the software

58

1. Design

5. QA 2. Build

4. Release 3. Configure

Tuesday, July 23, 13

59

1. Design

5. QA 2. Build

4. Release 3. Configure

Tuesday, July 23, 13

Tuesday, July 23, 13

Test Driven Development

Tuesday, July 23, 13

Test Driven Development

Behavior Driven Development

Tuesday, July 23, 13

Test Driven Development

Behavior Driven Development Domain Driven Development

Tuesday, July 23, 13

Test Driven Development

Behavior Driven Development

Application Bliss

Domain Driven Development

Tuesday, July 23, 13

POLYGLOT PERSISTENCE

alex brandtsr developer

Tuesday, July 23, 13

62

Tuesday, July 23, 13

63

Tuesday, July 23, 13

64

Tuesday, July 23, 13

65

RDBMS may not be the best fit

Tuesday, July 23, 13

66

Tuesday, July 23, 13

67

Tuesday, July 23, 13

68

5 Characteristics

Tuesday, July 23, 13

69

Non-relational

Tuesday, July 23, 13

70

Distributed

Tuesday, July 23, 13

71

Open-source

Tuesday, July 23, 13

72

Horizontally Scalable

Tuesday, July 23, 13

73

Schema-less

Tuesday, July 23, 13

orders

customers

order lines

credit cards

ID: 1001

line items:

customer: Ann

0321293533

0321601912

0131495054

2

1

1

$48 $48

$39

$51

$39

$51

payment details:

Card: AmexCC #: 12345expires: 04/2001

74

Tuesday, July 23, 13

75

There are several different types of data stores that are considered to

be NoSQL

Tuesday, July 23, 13

76

Key/Value Databases

Tuesday, July 23, 13

77

Document Databases

Tuesday, July 23, 13

78

Column-Family Stores

Tuesday, July 23, 13

79

Graph Databases

Tuesday, July 23, 13

80

Types in Detail

Tuesday, July 23, 13

Key/Value DatabaseS

81

Tuesday, July 23, 13

Key/Value DatabaseS

• Good Use Cases:

• Storing Session Information

• User Profiles & Preferences

• Shopping Cart Data

• Bad Use Cases:

• Data Relationships

•Multi-operation Transactions

•Query by Data

82

Tuesday, July 23, 13

Key/Value DatabaseS

• Good Use Cases:

• Storing Session Information

• User Profiles & Preferences

• Shopping Cart Data

• Bad Use Cases:

• Data Relationships

•Multi-operation Transactions

•Query by Data

83

Tuesday, July 23, 13

document Database

84

Tuesday, July 23, 13

document Databases

•Good Use Cases:

• Event Logging

• Blogging Platforms

• Real-time Analytics

• Article storage

• Bad Use Cases:

• Complex Transactions

•Queries against Varying Aggregate Structures

85

Tuesday, July 23, 13

document Databases

•Good Use Cases:

• Event Logging

• Blogging Platforms

• Real-time Analytics

• Article storage

• Bad Use Cases:

• Complex Transactions

•Queries against Varying Aggregate Structures

86

Tuesday, July 23, 13

COLUMN FAMILY

87

Tuesday, July 23, 13

88

Tuesday, July 23, 13

COLUMN-FAMILY datastores

•Good Use Cases:

• Event Logging

• Blogging Platforms

• Counters (page visits)

• Expiring Usage

• Bad Use Cases:

• ACID transactions required

• Early prototype applications

89

Tuesday, July 23, 13

graph database

90

Tuesday, July 23, 13

Amanda

KNOWS

KNOWS

KNOWS

Robert

Gerry

Lamar

KNOWS

App 1

BUILTBUILT

KNOWS

App 2

USES

BUILT

Christie

KNOWS

Destiny

Jessica

KNOWS

App 3

BUILT

USES

USES

USES

KNOWS

graph database

91

Tuesday, July 23, 13

graph databases

•Good Use Cases:

• Connected Data

• Location-based services

• Recommendation engines

• Bad Use Cases:

• Ability to update specific aggregates

92

Tuesday, July 23, 13

Companies using nosql

93

DATABASE COMPANIES

Tuesday, July 23, 13

THE FUTURE IS POLYGLOT

94

Neo4Jdatabase

Cassandradatabase

Couchdatabase

MongoDBdatabase

RELATIONALDATABASE

Graph-structureddomain rules

Columnar dataaccess with

decentralization

Documentstructures

Document structureswith offlineprocessing

(Actors) (Actors)

(Actors)(Actors)

Asynchronous message passing

MODULE 1

MODULE 2

MODULE 3

MODULE 4

IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

Tuesday, July 23, 13

95

POLYGLOT PATTERN: cache

ORDERTAKING

RESTAURANTMANAGEMENT

REDISCACHE

MYSQL DATABASE SECONDFIRST

CONSUMERRESTAURANT

OWNER

Credit: @crichardson

Tuesday, July 23, 13

96

POLYGLOT PATTERN: write to sql and NOSQL

NEWTWEET

INSERT

LPUSHXLTRIM

MySQL

TWEETS

FOLLOWS

FOLLOWERS

Redis: Timeline

FOLLOWER 1

FOLLOWER 2

FOLLOWER 3

Avoidexpensive

MySQL joins

Credit: @crichardson

Tuesday, July 23, 13

97

POLYGLOT PATTERN: REPLICATE FROM MYsql to NOSQL

QUERYSERVICE

UPDATESERVICE

REDISMYSQL

DATABASE

SYSTEMOF RECORD

MATERIALIZED VIEW

READER WRITER

replicate and

denormalize

query() update()

Credit: @crichardson

Tuesday, July 23, 13

98

Atomic

Consistent

Isolated

Durable

ACID to base

Basically Available

Soft state

Eventually consistent

Tuesday, July 23, 13