oak , the architecture of the new repository
DESCRIPTION
Oak , the Architecture of the new Repository. Michael Dürig, Adobe Research. Design goals. Scalable Big repositories Clustering Customisable, flexible OSGi friendly. Outline. CRUD Changes Search. Tree model. a. d. b. c. Updating. ?. a. d. b. c. x. MVCC. HEAD. r1: /. - PowerPoint PPT PresentationTRANSCRIPT
APACHE SLING & FRIENDS TECH MEETUPBERLIN, 22-24 SEPTEMBER 2014
Oak, the Architecture of the new RepositoryMichael Dürig, Adobe Research
Design goals
adaptTo() 2014 2
Scalable Big repositories Clustering Customisable, flexible OSGi friendly
Outline
adaptTo() 2014 3
CRUD Changes Search
Tree model
adaptTo() 2014 4
a d
b c
Updating
adaptTo() 2014 5
?
x
a d
b c
MVCC
adaptTo() 2014 6
a d
b c
r2: /dr1: /d
r1: /a/br2: /a/b
r1: / r2: /
r2: /d/x
HEAD
adaptTo() 2014 7
Refresh and Garbage Collection
garbage
Refresh
adaptTo() 2014 8
garbage
Garbage collection
adaptTo() 2014 9
adaptTo() 2014 10
Concurrency and Conflicts
r2a
Concurrent updates
adaptTo() 2014 11
r2br1
Merging
adaptTo() 2014 12
r1 r3merge
r2b
r2a
upates
Conflict handling: serialisation
adaptTo() 2014 13
Fully serialised Fail, no concurrent update
Partially serialised Concurrent conflict free updates
Conflict handling strategies: merging
adaptTo() 2014 14
Partial merge Conflict markers, deferred resolution
Full merge Need to choose victim
adaptTo() 2014 15
Replicas and Sharding
Replica and caches
adaptTo() 2014 16
master copy full replica cache
Sharding strategies
adaptTo() 2014 17
by path by level by hash with caching
adaptTo() 2014 18
Implementations
MicroKernel / NodeStore
adaptTo() 2014 19
Tree / Revision model implementation
Responsible for
Clustering
Sharding
Caching
Conflict handling
Not responsible for
Validation
Access control
Search
Versioning
Current implementations
adaptTo() 2014 20
DocumentMK TarMK (SegmentMK)
Persistence MongoDB, JDBC Local FS
Conflict handling Partial serialisation Full serialisation
Clustering MongoDB clustering Simple failover
Sharding MongoDB sharding N/A
Node Performance Moderate High
Key use cases Large deployments (>1TB), concurrent writes
Small/medium deployments, mostly read
adaptTo() 2014 21
Access Control
Accessible paths
adaptTo() 2014 22
a d
cb
xistentialism
adaptTo() 2014 23
All paths traversable Node may not exist Decorator on NodeStore
E
root.getChildNode("a").exists();
root.getChildNode("a") .getChildNode("b").exists();
⟹ false
⟹ true
adaptTo() 2014 24
Comparing Revisions
Content diff
adaptTo() 2014 25
What changed between trees Cornerstone for
Validation Indexing Observation …
What changed?
adaptTo() 2014 26
∆
Example: merging
adaptTo() 2014 27
r1
r2a
r2b
r3
r1 ➞ r2a“a” modified
“b” removed
∆
r1 ➞ r2b “d” modified
“x” added
∆
adaptTo() 2014 28
Commit Hooks
Commit hooks
adaptTo() 2014 29
Key plugin mechanism Higher level functionality
Validation (node type, access control, …) Trigger (auto create, defaults, …) Updates (index, …)
Editing a commit
adaptTo() 2014 30
∆ ∆ + x
Commit hooks
adaptTo() 2014 31
Based on content diff pass a commit fail a commit edit a commit
Applied in sequence
Type of hooks
adaptTo() 2014 32
CommitHook Editor Validator
Content diff Optional Always Always
Can modify Yes Yes No
Programming model
Simple Callbacks Callbacks
Performance impact
High Medium Low
adaptTo() 2014 33
Observers
Observers
adaptTo() 2014 34
Observe changes After commit Often does a content diff Asynchronous Optionally synchronous
Local cluster node only
Examples
adaptTo() 2014 35
JCR observation External index update Cache invalidation Logging
adaptTo() 2014 36
Search
Query Engine
adaptTo() 2014 37
SELECTWHERE x=y
/a//*
Parser
Parser
Parser
Index
Index
Traverse
Parser
Index
parse execute post process
Index Implementations
adaptTo() 2014 38
Property (ordered) Reference Lucene
In-content or file system Solr
Embedded or external
adaptTo() 2014 39
Big Picture
Big picture
adaptTo() 2014 40
MicroKernel
Oak Core
Oak JCR
Oak API
NodeStore API
JCR API
Plugins
Resources
adaptTo() 2014 41
http://jackrabbit.apache.org/oak/
adaptTo() 2014 42
Appendix
Resources
adaptTo() 2014 43
http://jackrabbit.apache.org/oak/
http://jackrabbit.apache.org/oak/docs/
https://svn.apache.org/repos/asf/jackrabbit/oak/trunk
/