How to evolve your analytics stack with your business using Snowplow

Download How to evolve your analytics stack with your business using Snowplow

Post on 12-Apr-2017

271 views

Category:

Data & Analytics

1 download

Embed Size (px)

TRANSCRIPT

  • EVOLVING YOUR ANALYTICS STACK WITH YOUR BUSINESS

    SNOWPLOW - LONDON MEETUP #4

  • SNOWPLOW - LONDON MEETUP #4

    BUSINESSES ARE CONSTANTLY EVOLVING

    Your products (apps & platforms) change

    Your questions should change too

    Its critical that the analytics stack can evolve with your business

  • SNOWPLOW - LONDON MEETUP #4

    SELF-DESCRIBING DATA EVENT DATA MODELING+

    EVOLVING EVENT DATA PIPELINE

    HOW?

  • SELF-DESCRIBING DATAPART 1

  • SNOWPLOW - LONDON MEETUP #4

    NO TWO COMPANIES ARE ALIKE

  • SNOWPLOW - LONDON MEETUP #4

    DEFINE YOUR OWN EVENTS AND ENTITIES

    Events

    Entities

    Build castle

    Form alliance

    Declare war

    Player

    Game

    Level

    Castle

    View product

    Buy product

    Deliver product

    Product

    Customer

    Basket

    Vehicle

  • "description": "Schema for a fighter context",

    "vendor": "com.ufc",

    "name": fighter",

    "version": 1-0-2,

    "properties": {

    "FirstName": {"type": "string"},

    "LastName": {"type": "string"},

    "Nickname": {"type": "string"},

    "FacebookProfile": {"type": "string"},

    "WeightLbs": {"type": ["integer", "null"]},

    "Record": {"type": string", "pattern": "^[0-9]+-[0-9]+-[0-9]+$"}

    }

    }

    SNOWPLOW - LONDON MEETUP #4

    YOU THEN DEFINE A SCHEMA FOR EACH EVENT AND ENTITY

    I DONT DO EVENTS THAT ARENT SCHEMAED

  • SNOWPLOW - LONDON MEETUP #4

    YOU THEN DEFINE A SCHEMA FOR EACH EVENT AND ENTITY

    "schema": "iglu:ufc/fighter/jsonschema/1-0-2",

    "data": {

    "FirstName": Daniel

    "LastName": Cormier,

    "Nickname": DC,

    "FacebookProfile": Daniel-Cormier,

    "TwitterName": dc_mma,

    "WeightLbs": 205

    }

    }

  • SNOWPLOW - LONDON MEETUP #4

    THE SCHEMAS CAN THEN BE USED IN A NUMBER OF WAYS

    Validate the data (important for data quality)

    Load the data into tidy tables in your data warehouse

    Make it easy / safe to write downstream data processing application (e.g. for real-time users)

  • EVENT DATA MODELINGPART 2

  • SNOWPLOW - LONDON MEETUP #4

    WHAT IS EVENT DATA MODELING?

    Event data modeling is the process of using business logic to aggregate over event-level data to produce 'modeled' data that is simpler for querying.

  • SNOWPLOW - LONDON MEETUP #4

    MODELED VS UNMODELED DATA

    event 1

    event n

    Users

    Sessions

    Funnels

    IMMUTABLE. UNOPINIATED. HARD TO CONSUME. NOT

    MUTABLE AND OPINIONATED. EASY TO CONSUME.

  • SNOWPLOW - LONDON MEETUP #4

    IN GENERAL, EVENT DATA MODELING IS PERFORMED ON THE COMPLETE EVENT STREAM

    Late arriving events can change the way you understand earlier arriving events

    If we change our data models: this gives us the flexibility to recompute historical data based on the new model

  • EVOLVING THE DATA PIPELINEPART 3

  • SNOWPLOW - LONDON MEETUP #4

    HOW DO WE HANDLE PIPELINE EVOLUTION?

    Businesses change over time

    The events that occur are going to change

    Use of the data will change

    Insight -> more questions -> more insight -> more questions

    Two types of evolution: push and pull

    BUSINESSES ARE NOT STATIC, SO EVENT PIPELINES SHOULD NOT BE EITHER

  • SNOWPLOW - LONDON MEETUP #4

    PUSH EXAMPLE:

    If data is self-describing it is easy to add an additional sources

    Self-describing data is good for managing bad data and pipeline evolution

    IM AN EMAIL SEND

    EVENT AND I HAVE INFORMATION ABOUT THE

    RECIPIENT (EMAIL

  • SNOWPLOW - LONDON MEETUP #4

    ANSWERING THE QUESTION:

    1. EXISTING DATA MODEL SUPPORTS ANSWER

    2. NEED TO UPDATE DATA MODEL

    3. NEED TO UPDATE DATA MODEL AND DATA COLLECTION

  • SNOWPLOW - LONDON MEETUP #4

    SELF-DESCRIBING DATA AND THE ABILITY TO RECOMPUTE DATA MODELS ARE ESSENTIAL TO ENABLE PIPELINE EVOLUTION

    SELF-DESCRIBING DATA RECOMPUTE DATA MODELS ON ENTIRE DATA SET

    Updating existing events and entities in a backward compatible way e.g. add optional new fields

    Update existing events and entities in a backwards incompatible way e.g. change field types, remove fields, add compulsory fields

    Add new event and entity types

    Add new columns to existing derived tables e.g. add new audience segmentation

    Change the way existing derived tables are generated e.g. change sessionization logic

    Create new derived tables

  • QUESTIONS?

    SNOWPLOW - LONDON MEETUP #4