Building Stream-Analytic Ecosystems for Enterprises

Approved/External Publishing/Raman Ganesh

Building Stream-Analytic Ecosystems for Enterprises1

AGENDA

• Part I : Ecosystem needs of an Enterprise for Stream Analytics

• Part II : Introduction to Meta-programming

• Part III : Metaprogramming on-top of Apex: Demo

• Part IV : Agility For Analytics In Enterprises

• Part V : How it is different from Apache Beam Model?

2

Part I — Ecosystem needs of an Enterprise for Stream Analytics

3

Ecosystem needs of an Enterprise for Stream Analytics

4

Type systems composition for

Domain Validations

Individuals, Interactions,Platforms,Processes

and Iterations

Ideation to realization latencies

and information exchange

End to end revision management

Domain refactoring

Integration, sandboxing, prototyping Productively

Managing different Semantic Models

Formal Semantics for Domains

5

Denotational Semantics

Operational Semantics

Axiomatic Semantics

And their sub-sets

Part II — Introduction to Metaprogramming

Building Stream-Analytic Ecosystems for Enterprises6

Metaprogramming

7

A metaprogram is a program that generates other programs or program parts

Code is data and Data is Code

Metaprogramming can be used to move computations from run-time to compile-time, to generate code using compile time computations, and to enable self-modifying code.

Usual types of metaprogramming :1- Interpreter integrated metaprogramming e.g. byte-code rewrite for JVMs

2-Dynamic expression evaluation from specific language ecosystems3-General purpose program transformation

Language-oriented programming

8

Language-oriented programming (LOP) is a style of computer programming in which, rather than solving problems in general-purpose programming languages, the

programmer creates one or more domain-specific languages for the problem first, and solves the problem in those languages

The concept of language-oriented programming takes the approach to capture requirements in the user's terms, and then to try to create an implementation

language as isomorphic as possible to the user's descriptions, so that the mapping between requirements and implementation is as direct as possible.

Part III — Metaprogramming on-top of Apex

Building Stream-Analytic Ecosystems for Enterprises9

An Interactive MPS preview of a DSL built ontop of Apex

10

Domain model only concerns user models

Operators are type-safe and formally verifiable with custom type rules

and structured

Comparison of the stock Apex CSVOp example in

Java vs DSL

Denotational semantics

Execution Semantics

Meta-concepts around the demo

Part IV : Agility For Analytics In Enterprises

Building Stream-Analytic Ecosystems for Enterprises11

Domain Driven Analytic ecosystems

12

Facilitating iterations : productively

Process built on a platform

Change latency for rapid analytic requirements

Ideation to realization latencies

Standardized workflows placing

Individuals and Interactions at centre

Common ecosystem problems solved with a meta-programming

platform!

Part V : How it is different from Apache Beam Model?

Building Stream-Analytic Ecosystems for Enterprises13

Beam model vs Metaprogramming approach

14

Beam model : Domain is data flow pipelines

Domain is the actual business domain

Denotationally hides underlying runners towards semantic

encapsulation but exposes Beam semantics externally

Denotationally hides underlying platforms and exposes domain semantics to the highest levels

Does not target analytic eco-system problem but only analytic platform

eco-system

Solves the meta-level problems for an analytic ecosystem and can easily

compose on top of Beam model denotationally

Questions

15