using couchbase and xdcr for real-time advertising applications: couchbase connect 2014
DESCRIPTION
Mirror Image provides a fully managed, globally distributed and load balanced real-time platform specifically designed for online and mobile advertising applications. Our customers’ advertising applications must execute with very fast response times, typically under 200 ms., regardless of where the request originates geographically. Some of these applications require very large data sets, with hundreds of millions of rows and hundreds or thousands of values per row. Until recently, since geographically distributed database consistency was so challenging to achieve, our applications managed customers’ data sets using flat files which were limited in size, impacting the use cases that we were able to implement. Learn how Couchbase Server with XDCR is enabling Mirror Image to work with much larger data sets to successfully implement and execute a wider range of advertising technology use cases for our customers.TRANSCRIPT
![Page 1: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/1.jpg)
Joe Lichtenberg
VP Advertising and Analytics, Mirror Image
Couchbase Connect
October 21, 2014
Using Couchbase and XDCR for Real-Time Advertising Applications
![Page 2: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/2.jpg)
Media Delivery
Content Logic
Mirror Image 1.0 (c. 1997): Content Delivery Network
![Page 3: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/3.jpg)
Media DeliveryEdge Computing
Content LogicContent Logic
Mirror Image 2.0: Edge Computing + Media Delivery
![Page 4: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/4.jpg)
Media DeliveryEdge Computing
Content LogicContent Logic
Dynamic Delivery: Edge Computing + Media Delivery
![Page 5: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/5.jpg)
Synchronized Operations Worldwide
![Page 6: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/6.jpg)
Obligatory Marketing Slide
Extensive functional capabilities
Personalized expert customization and support
High capacity, worldwide, real time infrastructure
Edge Computing
Geo-Distributed Database*
Live and On-Demand Video
Object Delivery
SSL Delivery
Token-Based Access Control
Reporting and Analytics
Knowledgeable, dedicated,in-house support team
Fully staffed, 24 x 7 Network Operations Center (NOC)
Expert professional servicesresources
High capacity, centralizedserver model
Exceptional performance, scalability, and availabilitywith SLA guarantees
Elastic scaling within and across geographies
Worldwide coverage
15+ years of experience
Mirror Image’s Dynamic Delivery Network
powers hundreds of billions of real-time requests for mission critical applications
throughout the Advertising and Advertising Technology ecosystem
![Page 7: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/7.jpg)
Why Focus on Advertising and Ad-Tech?
-Lots of prospects-Requires fast response times to each visitor’s browser / device
![Page 8: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/8.jpg)
Ad Tech Ecosystem
Source: Luma Partners
![Page 9: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/9.jpg)
Source: www.iab.net/data
Ad TechEcosystem – Another View
![Page 10: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/10.jpg)
Request/Response/Feedback Cycle
• Request Logic
– Customized behavior based on explicit and derived request attributes such as IP
address, user-agent, query parameters, cookie values, geo-location, …
• Response Logic
– Personalized javascript, XML, HTML; Transparent pixel GIFs; Cookie modifications;
Transformations, token substitutions…
• Edge Data Sets
– Shared data sets such as IAB Spiders & Bots, WURFL (Device DB), IP-GeoLocation
– Customer’s Key-Value Data
• Customized Data Collection
– Delivery of log files to customers as part of feedback cycle
![Page 11: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/11.jpg)
Edge Computing Flow
• IAB Spiders & Bots Data• Mobile Device Data Sets
• IP – Geolocation Data Sets• Custom Key-Value Data In Memory*
![Page 12: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/12.jpg)
What Problems Does The CS Implementation Solve?
• Customers and prospects have requirements that we workwith their large and growing data sets in real-time at theedge of the internet
• Our “flat file” implementation required moving the entirecontents of the files into memory on each server at startup,and…
• … customers were bumping up against data size limitations
![Page 13: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/13.jpg)
Edge Computing Flow
• IAB Spiders & Bots Data• Mobile Device Data Sets
• IP – Geolocation Data Sets• Custom Key-Value Data In Memory
• Couchbase Key-Value Distributed Database
![Page 14: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/14.jpg)
• Real-time GUID database
• Real-time cookie matching
• Contextual targeting
• Ad fraud / brand safety
• Cross-device user matching / audience de-duplication
• In session execution of batch-developed analytics
• Any web or mobile database-backed application with geographically dispersed users requiring fast end-to-end response times
Geo-Distributed Database: Use Cases
![Page 15: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/15.jpg)
Geo-Distributed Database: Requirements
• High performance lookup and write capabilities at the edge
• Ability to manage large custom data sets for each customer application
• Low latency replication (aka XDCR)
• Ability to replicate to different regions per application / different data per region
• Key-value lookups only– no need for complex queries or SQL
• Reliability
![Page 16: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/16.jpg)
Implementation Details
• Master hub sites have a 3 node Couchbase cluster for
reliability
• Edge sites start with a single instance for real-time RW
access, and scale out via clustering based on demand
• Edges can failover to nearby edges for availability
• Limited number of buckets
– Multi-tenant
– Geographic regions
– Customer-specific bucket(s) are optional
![Page 17: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/17.jpg)
Implementation Details
• Phase 1 (in production): Read-only edges; master
hubs; data import / ETL servers
• Phase 2: Extends read-only behavior to virtual
locations
• Phase 3: Adds write support on edges, bi-
directional XDCR back to each master hub
![Page 18: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/18.jpg)
GDD “Marchitecture” - Read Only
![Page 19: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/19.jpg)
GDD “Marchitecture” - Read/Write
![Page 20: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/20.jpg)
Master Hub Configuration
CustomizableETL Servers
FTP UploadStorage
CouchbaseEdge Instance
ETL MasterCouchbase 3-Node Cluster
ECF Server Farm
![Page 21: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/21.jpg)
Distributed Star Network Architecture
![Page 22: Using Couchbase and XDCR for Real-Time Advertising Applications: Couchbase Connect 2014](https://reader034.vdocuments.net/reader034/viewer/2022042715/559817911a28ab741d8b47c1/html5/thumbnails/22.jpg)
• Globally distributed, consistent data -> High performance globally
• Schema-less data model -> Flexibility
• Scalable -> Handles traffic and data growth and spikes
• Fully managed service -> Including setup, maintenance, replication, synchronization, backup, security, and 24x7 monitoring
• Subscription-based billing model -> No CAPEX, pay as you go
• Uses Couchbase Server -> Proven; active developer community
• Data privacy -> No conflicts of interest with our customers
GDD Key Benefits (or “Marketing Slide #2”)