maximizing your marklogic and java investments · maximizing your marklogic and java investments...

13 June 2017© COPYRIGHT MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Maximizing Your MarkLogic and Java InvestmentsScott A. Stafford, Principal Sales Engineer, MarkLogic

SLIDE: 2 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Photo attributed to smittenkitchen.com

SLIDE: 3 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Photo attributed to smittenkitchen.com

SLIDE: 4 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Photo attributed to disgruntled husband who had to clean oven

SLIDE: 5 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

MarkLogic Java Recipes§ Build and Deploy

§ Run automated testing

§ Load data from a relational database

§ Demo

BAM!

SLIDE: 6 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

SLIDE: 7 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

REST API

SLIDE: 8 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

REST API

JAVA CLIENT API

Build and Deploy

SLIDE: 10 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Roxy v1.0 v1.5 v1.75

2 0 1 71 2 0 1 0 2 2 0 1 1 3 2 0 1 2 4 2 0 1 3 5 2 0 1 4 6 2 0 1 5 7 2 0 1 6 8 2 0 1 7

Mgmt API 1.0(Read)

MarkLogic

ClientAPI 1.0

Mgmt API 2.0(Read/Write)

Mgmt API 2.0(Read/Write)

v4.1 v5 v6 v7 v8 v9

Java Client API v1.0 v2.0 v3.0 v4.0

Mgmt API 2.0(Read/Write)

Content Pump v1.0 v7 v8 v9

CORB v2.0 v2.2 v2.3.2

v2.6v1.0

v1.0

v2.6v1.0

ml-app-deployer

ml-gradlle

SLIDE: 11 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

HTTP Wrapper

Management REST API

SLIDE: 12 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

HTTP Wrapper

Configuration Coordinator

Management REST API

SLIDE: 13 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

#!/bin/bash

Management REST API

SLIDE: 14 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

MarkLogic Python API

Management REST API

SLIDE: 15 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

ml-app-deployer<java>

Configuration Coordinator

Management REST API

SLIDE: 16 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

ml-app-deployer<java>

Management REST API

SLIDE: 17 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

ml-app-deployer

Management REST API

ml-gradlejava application maven

Run automated testing

SLIDE: 19 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Single Deploy Configuration

cookbook-content

cookbook:8200

cookbook-modules

SLIDE: 21 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Single Deploy Configuration

cookbook-content

cookbook:8200

cookbook-modulesXML

SLIDE: 22 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Single Deploy Configuration

cookbook-content

cookbook:8200

cookbook-modulesXML XML

SLIDE: 23 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Multiple Deploy Configuration

cookbook-content

cookbook:8200

cookbook-modules

cookbook-test-content

cookbook-test:8201

TESTDELETEREPEAT

Local Deploy CI Deploy

SLIDE: 24 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Use Spring to inject DatabaseClient

Remove documents from database before test

execution

Load data from a relational database

SLIDE: 26 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

CONTENT PUMP

SLIDE: 27 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

SLIDE: 28 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Performance FailoverStatus Reliable

SLIDE: 29 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Basic Job Life-Cycle

DataMovement-Based Java Application MarkLogic Cluster

Listeners

Job ThreadsCreate, Configure, Start

Batcher

Listeners-sepcificaction

Work

Respose

ML Host

ML Host

ML HostNotifications

Work

ResponseBatcher

SLIDE: 30 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Demo Time!

SLIDE: 32 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

RECAP – MarkLogic Java Recipes§ Build and Deploy

§ Run automated testing

§ Load data from a relational database

SLIDE: 33 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

https://github.com/sastaffordsstafford@marklogic.com

Scott Stafford

Backup/Brainstorm

SLIDE: 35 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

MarkLogic Architectures§ Three Tier

- Spring Boot Powered – Front End, Spring MVC, MarkLogic

§ Orchestration Framework – ingestion/streaming

- MarkLogic Spring Batch

§ Microservices

- Apache Kafka + MarkLogic

SLIDE: 36 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Continuous InnovationROADMAP

Cerisent XQE Server 1 § ACID transactions§ Text-based search§ Linux support§ Web-based protocols§ HTTP and XDBC § XQuery§ XDBC support§ Role-based security

2003

2004

2005

2006 2010

2011

2012

2008 2013

2015

MarkLogic 2§ Clustering§ Document collections§ Enhanced search

(stemming, thesaurus, wildcard)

§ WebDAV support§ Document locking§ Backup and restore

MarkLogic 3§ Advanced Search§ Content Processing

(including PDF, Word, Excel, PPT)

§ HTTP calls§ Support for Linux,

Windows Server, .NET§ Facets, co-occur., value

lexicons, bucketing§ Failover

MarkLogic 3.1§ Wildcard queries§ Directories§ Forward compatibility§ XML Contentbase

Connector (XCC)

MarkLogic 4§ Alerting§ Entity enrichment§ Geospatial§ Modular documents§ Security auditing§ HA: forest-level

failover

MarkLogic 4.1-2§ Replication§ Failover§ Database Rollback§ Compartment Security§ Search Optimizations§ Search API§ REST capabilities§ SSL support§ Schema Validation§ Japanese added

MarkLogic 5§ Database Replication§ Multi-statement and

distributed transactions§ Point-in-time recovery§ Search API enhanced§ Monitoring

API/Dashboard§ Hadoop Integration

MarkLogic 6§ Accessibility§ SQL/BI§ Java/REST/JSON§ UDFs/Analytics§ mlcp§ Hadoop updates

MarkLogic 8§ JSON storage§ Server-side JavaScript§ Semantics§ Bitemporal§ Samplestack§ Java Client API§ Node.js Client API§ Management API§ Incremental backup§ Flexible replication§ Enhanced HTTP

Server

MarkLogic 7§ Semantics foundation§ Next-gen infrastructure

support § Elasticity§ Tiered storage§ Run on HDFS

SLIDE: 37 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

CI Process1. Deploy container with fresh MarkLogic install

2. Run gradle commands for MarkLogic initialization (mlInit, mlInstallAdmin)

3. Load application system data (taxonomies, lookups, system data)

4. Deploy container with middle tier (i.e. Spring Boot, nginx, nodejs)

5. Run tests (gradle test)

6. Run user interface tests

One HTTP request per document to a

single host

DMSDKDMSDK

Application Context

Processor

Writer

Reader

Application Context

INITIALIZATION

POST PROCSESING

READER

PROCESSOR

WRITER

SLIDE: 43 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

[“hip”, “hip”]

SLIDE: 45 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

SLIDE: 46 © COPYRIGHT 2017 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Get a gallon of milk

If (eggs == true) {

null;

}

Get a dozen

Some people, when confronted with a problem, think, 'I know, I'll use threads' - and then two they

hav erpoblesms.

Immutability changes everything

Knock knock.Race condition.Who's there?

There are only two hard problems in distributed systems:

2. Exactly-once delivery 1. Guaranteed order of messages

2. Exactly-once delivery