Redis in Kenshoo microservicesJesque based delayed priority queue system

Presented by Dima Vizelman

A Redis Use-Case

Performance Aggregation Microservice

So what I am going to speak about?

❏ Brief intro to Analysis App❏ Aggregation server requirements❏ Proposed solution❏ Solution Overview❏ Key features of the provided service

Brief intro to Analysis App

1. Analysis app

Orchestrates advertising across search platforms

Brief intro to Analysis App

2. Performance aggregation microservice

A micro service which performs aggregations of statistics\performance data to be used for analysis.

Aggregation request

Brief intro to Analysis App

Performance aggregation microservice

Aggregation types:

● On demand\user initiated or scheduled ● Lightweight or heavy ● Real time or delayed

Brief intro to Analysis App

❏ Brief intro to Analysis Apps❏ Aggregation server requirements❏ Proposed solution❏ Solution Overview❏ Key features of the provided service

● 500 Analysis Apps

● 2500 scheduled aggregations per App

● 200 user requests every 5 seconds

Aggregation server requirements

● UX

● Failure Handling

● Work in a cluster

Aggregation Server Requirements

❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service

Proposed Solution

● Priority Queues

● Delayed Jobs

● Distributed workers

Proposed Solution

● Simple

● Fast

● Robust

Jesque - why?

❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service

Solution Overview

❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service

Priority Queue

● No Priority Jobs

● High/Low Priority Queues

● Workers vs Queues

Priority Queue

Why delayed jobs ?

Delayed Jobs

Delayed Jobs

❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service❏ Bonus features

Disaster Recovery

1. Distributed cache

2. Persistency for disaster recovery information

Q&A

Thank You!