forward: declarative ajax web applications through sql++ on a unified application state

1

FORWARD:Declarative Ajax Web

Applications through SQL++ on aUnified Application State

Yupeng Fu Kian Win Ong

Yannis Papakonstantinou

UC San Diego

2

• Web application framework

• Single language: SQL++

• Zero frictions and automatic optimizations– Enable IT personnel to create Ajax apps and interactive

visualizations– Improve development productivity

• Alpha release– http://forward.ucsd.edu– 8 applications (commercial + academic)

FORWARD

3

FORWARD Website

http://forward.ucsd.edu

4

Simple Web Application

5


6


7

Mundane Low-Level Programming for Ajax Applications

Browser

ApplicationServer

Database

DOM (XHTML),JavaScript (JSON)

Java, Pythonetc.(Objects)

SQL (Tables)

Event Side-effect

Convert JSON to objects

Convert objects to tables

Convert tables to objects

Convert objects to JSON

Challenge #1: Impedance mismatch due to different languages and data models

Requires using multiple languages, tedious data modeling and coding

8


Browser

ApplicationServer

Database



SQL (Tables)

Event Side-effect

Retrieve user input

Serialize and parse request

Check access rights

Parameterize SQL query

INSERT INTO ...

Challenge #2: Distributed data access and

programming due to multiple tiers and machines

Requires partitioning code manually, and ad-hoc

coordination across multiple machines

9


Browser

ApplicationServer

Database



SQL (Tables)

Event Side-effect

SELECT *FROM ...WHERE id = ...

Instantiate partial HTML and JSON template

Incrementally update old state to new state

Incrementally re-render DOM and JavaScript components

Challenge #3: Incrementally modifying old page to new page due to Ajax programming model

Requires identifying data flow dependencies, correctly

transitioning from one consistent state to another.

10

Incremental Updates are Error-Prone

• New requirements:• Libraries with saved

comments are green instead of red

• Multiple code paths to change:• Code that loads the page• Code that saves the

comment

• Code complexity increases with number of events/actions

11

FORWARD Architecture

Actions

Unified Application State (UAS)

Pages

• SQL++ as single language (inspired by Hilda and Links)

• SQL extensions for semi-structured features (nesting, heterogeneity)

• State machine conceptual model• No need for multiple tiers, machines, languages

12

Runtime Interpreter


Actions


Pages

read writesideeffects next_page

http://localhost/libraries

• Virtual database of different data sources

• Unifying data model

• PL/SQL++

• INSERT, UPDATE, DELETE statements on UAS

• Control flow (conditionals, loops, functions)

13

Runtime Interpreter


Actions


Pages

read

Page instance

rendering• Declarative page as rendered queries

• Data of page is a SQL++ query over UAS

• Rendering of page is template markup

• Template markup for both HTML and rich JavaScript components

• No side-effects: can only read from UAS

14

Runtime Interpreter


Actions


Pages

read writesideeffects next_page

User Input

Page instance

renderingAjax (XHR)

read

• Action only saves comment

• Action does not specify how to replace the textarea and Save button on the page

15

FORWARD: Single Language, Location Transparent, Declarative

ActionSpecifications

Unified Application State (UAS)Specification

Page Specifications

• SQL++ as single language

• Challenge #1: Impedance mismatch

• Declarative page as rendered queries

• Challenge #3: Incrementally modifying old page to new page

• Virtual database

• Challenge #2: Distributed data access and programming

16

• Motivation

• Architecture

• Syntax and Semantics– Unified Application State– SQL++– Page– Action

• Future Work

Outline

17

UAS Specification


RDBMS(Tables)

Session(Objects)

URL Params(KV Pairs)

User Input(JSON)

define sql source db options { driver : 'postgresql', host : 'book_db_host', port : 5432, database : 'libraries', user : 'postgres', password : '9xk8NToA'}

Source Specification

SQL Source Wrapper

• Developer provides source specification

• Framework provides capability-based source wrappers that execute different subsets of SQL++

• SQL++ queries over UAS virtual database• Run by FORWARD distributed query processor

• Current sources: • SQL database• In-memory SQL++ values

• Future sources: • JSON databases (MongoDB, CouchDB)• Large-scale data services (BigQuery, Redshift)

select ...from db.librariesleft join session.notes on ...where ...

18

SQL++ Data Model

value = null | scalar | tuple | table

scalar = primitive | type "(" primitive ")"

tuple = "{" ( name : value "," )* "}”

table = "[" ( key? tuple "," )* "]"key = "(" primitive ")"

{ lat : 0.0 , lng : 12.3 , ... }

[ (1) { lat : 0.0 , lng : 12.3 } , (2) { lat : 0.0 , lng : 45.6 }]

12.3

date('2013-08-30')

• Extended JSON syntax with rich types and keys

• Opposite of Math conventions!

• Data model designed to capture both SQL tables and JSON for the need of pages• Captures subset of JSON used in

~30 JavaScript components

19

SQL++ Data Model

• Extensions over SQL tables: • Optional type checking• Root value• Nesting• Heterogeneity• Optional keys

• Extensions over JSON• Optional keys• Rich scalar types (e.g. dates)

• Work in progress• Order• Table of scalars/tables

• Analogous SQL++ extensions to query language

value = null | scalar | tuple | table

scalar = primitive | type "(" primitive ")"

tuple = "{" ( name : value "," )* "}”

table = "[" ( key? tuple "," )* "]"key = "(" primitive ")"

20

Page Instance

_1_html : { template : '<body>...', children : {

_2_google_map_Maps : { markers : [ { position : { lat : 32.7, lng : -117.2 },

infowindow : _3_html : { template : '<div>...', children : ... } } ... ] } }}

SQL++ Value

State of JavaScript component

HTML string template with placeholders for children

21

Page Instance

_1_html : { template : '<body>...', children : {

_2_google_map_Maps : { markers : [ { position : { lat : 32.7, lng : -117.2 },

infowindow : _3_html : { template : '<div>...', children : ... } } ... ] } }}

SQL++ Value

<% html %><body> ..

<% unit google.map.Maps %> { markers : [ { position : { lat : 32.7, lng : -117.2 },

infowindow : <% html %> <div> ... </div> <% end html %> } ... ] } <% end unit %></body><% end html %>

Syntactic sugar: inline HTML

Template Markup

Syntactic sugar: visual units

Also SQL++ query Also Page Specification

Yannis

there are two syntactic sugars here. The %unit also

22

• Motivation

• Architecture

• Syntax and Semantics– Unified Application State– SQL++– Page– Action

• Future Work

Outline

23

Page Specification – Static Page<% html %>...

<% unit google.map.Maps %> { markers : [ { position : { lat : 32.7, lng : -117.2 }, infowindow : <% html %> Library: Del Mar Library <br /> Comment: <% unit html.TextArea %> { value : Great! } <% end unit %> <br /> <button value="Save"> <% end html %> } ... ] } <% end unit %><% end html %>

• Declarative static page using only template markup• HTML• Visual unit wrappers for

JavaScript components

• Visual unit wrappers translate between state and behavior• Translation via renderers

and collectors• No JavaScript code is

needed on the page specification

Yannis

I am loosing what you do here. We should say up front that a page specification is a query with respective syntacti sugar with the page instance + a little more in order to align with template notations, to be seen next.

24

Page Specification – Dynamic Visualization<% with libraries as select * from db.libraries left join session.notes on ... where ...%>

<% unit google.map.Maps %>{ markers : [ <% for l in libraries primary key (library_id) %> { position : { lat : <%= l.lat %> lng : <%= l.lng %> }, ... } <% end for %> ]}<% end unit %>

• Declarative visualization page using only template markup and SQL++ queries

• Dynamic data of page specified by SQL++ queries over UAS

View

Syntactic sugar: iteration

Substitution

25

Page Specification – Dynamic Visualization<% with libraries as select * from db.libraries left join session.notes on ... where ...%>


with libraries as select ... from db.libraries left join session.notes on ... where ...

{ markers : select position : { l.lat as lat, l.lng as lng } from libraries as l primary key (library_id)}

• Incremental updates are delegated to FORWARD’s• Incremental renderers• Incremental view maintenance

26

Page Specification – User Input<% with libraries as select * from db.libraries left join session.notes on ... where ...%>


<% define comment string %>infowindow : <% html %> Library: <%= l.library_name %> <br /> Comment: <% if l.comment is null %>

<% unit html.TextArea %> { value : <% bind comment %> } <% end unit %> <br /> <button value="Save" <% event click ajax /save_note %>>

<% else %>

<%= l.comment %> <br /> <button value="Delete" <% event click ajax /delete_note %>>

<% end if %> <% end html %>

• Declarative page for visualization and user input

• User input is bound into the UAS

• Events trigger an action with its context

Variable in user input source of UAS

Binds to variable

Event / Action

Yannis

I do not believe that the audience will understand in what way:a. this is a WITH queryb. the "for" corresponds to an SQL query.We need to write the query explicitly.

27

Action Specification

define action /save_note function save_note;

define function save_note() asbegin insert into session.notes (library_ref, comment) values ( context.l.library_ref, context.comment ); next_page('map');end;

• An action is a URL mapped to a function (procedure) that causes side-effects

• Action can access all variables in the event’s context• define (mutable)• with (immutable)• for ... in (immutable)

• insert, update and delete statements writes to UAS

• next_page() chooses the next page to be evaluated and displayed

28

• Incremental view maintenance (IVM) of pages– Incremental update of pages is delegated to FORWARD’s optimizations– First IVM prototype by reduction to delta-based, relational IVM

• Y. Fu, K. Kowalczykowski, K. W. Ong, Y. Papakonstantinou and K. K. Zhao. Ajax-based Report Pages as Incrementally Rendered Views. SIGMOD 2010.

– Work-in-progress based on algebraic rewrite rules that propagate predicates (as opposed to data)

• Distributed query processor– Virtual database is enabled by FORWARD’s distributed query processor– Recent work led to distributor that is tuned for data of web applications: large

database and small in-memory objects

• Lightweight JavaScript components– Visual unit wrappers for components with 50++ attributes / methods require upfront

time and effort to translate between state and behavior– Work in progress: enable translation for subset of attributes / methods that matter

to a page

Related and Future Work in FORWARD

29

Cloud-based IDE

30

FORWARD Website

http://forward.ucsd.edu

forward: declarative ajax web applications through sql++ on a unified application state

Documents