stored procedure as a service

Stored Procedureas a

Service (SPaaS)

t @abhishektiwari w https://abhishek-tiwari.com

• solutions manager apac @ Isentia• leading media intelligence company• data and product engineering• data processing with NLP & search focus


Bit ofcontext

Wolf in sheep's clothing

• a data pipeline built using legacy technologies• database as queue, ETL using StoredProc• trigger calling StoredProc calling trigger• stateful, strongly coupled, monolith databases


• thin applications fat StoredProc VLDB• more than 5000 StoredProc written over 10Y• deliver new products/services in a short span• need to move fast - continuous delivery

Ourchallenge


subroutine stored in the database data dictionary

StoredProc

Application(VC) DatabaseClientUIInBrowser

StoredProc&

UDFs


StoredProc

good idea that quickly turned into massive technical debt


Moving data is harder than moving logic

StoredProc


Good Bad Ugly

StoredProcConcerns

• improved performance• security and access control

• highly procedural• limited constructs• can't pass objects• vendor lock-in/specific• maintainability

• zillion parameters• business logic• versioning & testing• debugging & logging• continuous delivery• cross-database StoredProc


Phased Approach


Ecosystem of MicroservicesDecouple UI and Backend Ecosystem of APIs

Separate UI layer from the backend layer


ClientUIInBrowser

APIs

Static Stateless UI(Angular/React)

Delivered via CDN

w https://example.com

w https://api.example.com

MobileApps

Convert legacy backend layer into ecosystem of APIs


Ecosystem of APIson top of Databases

Instant CRUD APIs without code

LoopBack Script Logic

DreamFactory Script Logic

Roll your own APIs

FaaS

Authentication & Authorization Setup

Script Logic

SPaaS Authentication & Authorization Setup

decision tree

SPaaS


APIGatewayClientUI APIs

(Node.js)w https://api.example.com

DatabaseStoredProc

&UDFs

MSSQL

StoredProc

Node.js

Express.js mssql

Middleware RouteHandler

connect

execute

TDSdriver

TDSProtocol

UDFs

HTTPRequest

/v1/coverage/today/50

HTTPResponse


Promises, StreamsorCallbacks

ConnectionPooling

Wiring up StoredProc

Execute

SQL2JSON

Variation of FaaS

• self-contained piece of reusable functionality• can be executed by events or API endpoints• read and write to database backend• can be written in multiple languages


SimilarConcerns

• vendor lock-in or specific implementations• tooling for continuous integration & delivery• monitoring, logging and debugging• testing, discovery, security and latency


SPaaS API Delivery

DevelopAPIs

Changecommitted

Build&Test

PromotetoUAT

PromotetoProd

UATServers

ProdServers

Artifacts


SPaaS SQL Delivery

StoredProcChanges

Changecommitted

Build&Test

PromotetoUAT

SchemaChanges

Changecommitted

PromotetoProd

DatabaseUAT

DatabaseProd

Artifacts

SQLServer

Managem

entStudio


Redgate SQL Compare

DatabaseINT

DatabaseDEV

Redgate DLM Automation

• mock API request & response using Swagger• mock environments as first-class citizen• reusable API tests – contract, load, security• tooling interoperability using Swagger

Mockfirst


Breaking Monolith

• strict rule - no new StoredProc • separate business logic & data logic• build microservices – one at a time• each microservice owns it’s own data store



w https://api.example.comDatabase

StoredProc&

UDFs

ServiceA DB-A

ServiceB DB-B

ServiceX DB-X

CDC

Transition



w https://api.example.com

ServiceD DB-D

ServiceE DB-E

ServiceX DB-X

CDC

Future


ServiceC DB-C

ServiceB DB-B

ServiceA DB-A

DrivingOutcomes

• more than 200 APIs in less than 6 months• two new native mobile products• a new responsive web product • seamless opportunities – API economy


• team maturity drives API maturity model• experience design influences API reusability• developer experience defines overall success• use Swagger as epicentre of you API program

LessonsLearned



Q&A

stored procedure as a service

Software