introduction to marklogic data hub service...17 june 2019© marklogic corporation introduction to...
TRANSCRIPT
17 June 2019© MARKLOGIC CORPORATION
Introduction to MarkLogic Data Hub Service
Caio MilaniSenior Director, Product Management, MarkLogic
Sebastian KalinowskiStaff Engineer, MarkLogic
Agenda
Overview of MarkLogic Data Hub Service
Quick look into Data Hub development
Data Hub Service demo
Q&A with the product team
Drivers for Better Data Integration
Unpredictable Spending
Cloud Data Sprawl
Lackof Agility
MarkLogic Data Hub Platform
ADVANCED SECURITY
SMART CURATION
UNIFIED PLATFORM
LOAD DATA AS IS SIMPLE DEVELOPMENT
Overview of Data Hub ServiceCloud service that deploys in minutes with predictable, low cost
Fast, Cost-Effective Data Integration
Cloud ServiceData Hub
Data Hub automation Cost-effective in the cloud Reliability in the cloud
Fast data integration and curation Simplicity of a unified platform Data governance and security
Much More Than a Database as a Service
On-Premises
DATA CENTERS
NETWORKING
STORAGE
SERVERS
VIRTUALIZATION
OS
DOCUMENT DB
GRAPH DB
RELATIONAL DB
SEARCH
ETL
MDM
SECURITY
APPS
DATA CENTERS
NETWORKING
STORAGE
SERVERS
VIRTUALIZATION
OS
DOCUMENT DB
GRAPH DB
RELATIONAL DB
SEARCH
ETL
MDM
SECURITY
APPS
DATA CENTERS
NETWORKING
STORAGE
SERVERS
VIRTUALIZATION
OS
DOCUMENT DB
GRAPH DB
RELATIONAL DB
SEARCH
ETL TOOLS
MDM
SECURITY
APPS
DATA CENTERS
NETWORKING
STORAGE
SERVERS
VIRTUALIZATION
OS
DOCUMENT DB
GRAPH DB
RELATIONAL DB
SEARCH
ETL TOOLS
MDM
SECURITY
APPS
IaaS DBaaS Data Hub Service
The most comprehensive out-of-the-box cloud service stack
HARMONIZATION HARMONIZATION HARMONIZATION HARMONIZATION
MarkLogic Data Hub Demo…
Data Hub Service
Fast Deployment
Remove skills, hardware, config and ops barriers that slow adoption
Cost-effectiveness
Predictable low cost of a subscription…even when demand
spikes
Reliability
Fully managed, secured and certified service based on a 100% ACID
database
Predictable Cost
- Users choose capacity level at a fixed cost (in MCUs)
- By hour or by year
‘Rollover Credits’
- Unused credits accumulate up to 12x the baseline
Auto-scale with a Predictable Cost
Finding Equilibrium With Cloud Credits
Credits are consumed whenever end-points are utilized for query, ingest, and curation
Maximum level that can be accumulated is 12x the hourly credit amount Credits are accumulated when operating
below baseline threshold
Credits are earned at the baseline rate
MCU CREDIT BALANCE
Finding Equilibrium With Cloud Credits
MCU CREDIT BALANCE
Bursting above baseline
Architecture
CUSTOMER LDAP
CUSTOMERVPC
VPCPEERING
Data Hub Service Architecture
INGESTION & CURATION ACCESS
SERVICE VPC(CUSTOMER ISOLATED)
LOAD BALANCER
LOAD BALANCER
Data
D-NODES
Operational AnalyticalCuration
LOAD BALANCER
MarkLogic Tools
DMSDK MLCP REST API
Data Hub Service Responsibilities
Fast Deployment
Provide all hardware and infra-structure
Automate provisioning
Automate backups
Cost-effectiveness
Choose the right hardware based on capacity requirements
Automate scalability based on usage and credits
Reliability
Always deploy in HA
Perform upgrades/patching
Monitor 24x7
Support
All databases and app servers are built-in
- Users cannot add/remove
All highly available
All forests, caches configured for optimal normal performance
Under The Covers: Databases and App Servers
How Security Works
Data Hub Framework
Ships with built-in roles
Customers have no admin access
Security admin can reuse these roles
Users are managed by customer LDAP
marklogicsvc.com console
Multiple services, multiple admins
Users are managed by security admin
Billing is managed by account admin
Flow Developer
Can upload new/changed documents(e.g. flows) to the modules database.
Flow Operator
Load and modify data in the staging databaseand final database, call flow runner
Endpoint Developer
Has access to endpoints and the final documents,can add documents to the modules database
Endpoint Operator Has access the endpoints
ODBC Has access to the analytics stack that has an ODBC server
Service Admin Can create and modify the service
Account Admin Can see billing, account details
Security Admin Can assign the DHF roles to other roles, and create vanilla roles
Creating a Data Hub Service…
Deploying a Data Hub Service…
Predictable, fixed costs for unpredictable, varying demand
Lower cost than AWS services
- AWS doesn’t have a data hub service, they have parts
- Add up the cost of the parts needed to assemble a data hub and it’s expensive
- And you’re doing all of the integration work – that makes its really expensive!
Simpler than “Free”
- With “free” software, you still pay AWS for servers, storage, etc.
- You also have to figure out how to deal with peaks and valleys of demand
- Data Hub Service does that so efficiently, you pay less than a normal AWS bill + “free” software
Data Hub Service is an Incredible Value
Do you have questions?
Join us at the booth
Sessions
- Credit Suisse: Data, Digitalization, and Disruption – Tuesday 4:25 PM
- Search and the MarkLogic Data Hub – Wednesday 11:35 AM
Data Hub Service Workshop on Thursday. Please check availability
How to Learn More
Thank you