(js) export your own webgl viewer
TRANSCRIPT
(Js) Export your own webGL Viewer
Firenze November 11, 2015
Alberto Mancini - Francesca Tosi
About us
FREELANCE - SOFTWARE DEVELOPER (WEB&MOBILE)- LINUX SYSADMIN- DEVOPS
#JOOINK TEAM #GDG-FIRENZE #INTEL SOFTWARE INNOVATOR#GWTCON
[email protected]+AlbertoMancini
Alberto Mancini
About usFrancesca Tosi
FREELANCE- WEB & MOBILE DEVELOPER- SOFTWARE ENGINEER - SOFTWARE ARCHITECT WITH A PASSION FOR
CLEAN CODE AND FINE TUNED DETAILS.
#JOOINK TEAM #GDG-FIRENZE #GDG-ANCONA#INTEL SOFTWARE INNOVATOR#GWTCON
[email protected]@francescatosi
About usComputer Vision& Augmented Reality
picshare.jooink.com
JSInterop for fun and profit
Aim
What we have to do
● Data is (already) managed and processed in a serverside workflow.
● Data, meshes with arttributes and some metadata, has proprietary format and is not ready for web (huge size, repeated, not really optimized)
● There’s already a web application (sort of) that drive end-users interaction, should be ‘augmented’ with a 3D viewer.
Aim
Final Application
That sould be:
● performant (low bandwidth, caching capable, lighting fast)
● extensible (new features are expected to pop up)
● mobile (at least ‘tablet’) ready
● embeddable
Medical use-case:Provide a self contained WebGL component in GWT
to be consumed in vanilla javascript.
What we need
---
What we need:
● Manage 3D models in a Browser
● Get User Gestures (all devices)
● Get data/model from server and manage them on client
● Compunting … WebWorkers
Manage 3D Models in a Browser
3D Models
Manage 3D Models
3D Models
Manage 3D Models
Manage 3D Models:
● WebGL on js → Three.js
● Three.js for GWT →
3D Models
Parallax3D
Parallax 3D:
● Mature project
● It has a lot of Controls* but … we need something different
→ Natural Controls
*What a control is ?
3D Models
Parallax3D
Parallax 3D:
● Mature project
● It has a lot of Controls but … we need something different
→ Natural Controls*
*What a Natural Control is ?
3D Models
Parallax3D
Parallax 3D:
● Rendering (the heavy part) is done via WebGL so does not really matter (performance wise) if you use parallax or threejs so if you measure poor performance isn’t parallax fault … is your !!
Get User Gestures (Web/Mobile)
User Gestures
---
● Mobile version of Viewer needs gestures recognition
● Adapting Hammer.js v 2.0.4 for GWT
○ JsInterop○ JSNI
https://github.com/jooink
User Gestures
---
private Manager() {if(!injected) {
injected=true;ScriptInjector.fromString(...Hammer...).setWindow(...).inject();
}}
Hammer res:@Source("resources/hammer.min.js")public TextResource hammer();
User Gestures
We wrapped all the gestures/recognizers:
● Pan● Pinch● Press● Rotate● Swipe● Tap● ...
User Gestures
---
● Demo
Provide a self contained component in GWT to be consumed in vanilla javascript.
Js-Interop
The opposite way
● Fill a <div/>
● the js developers want to construct the whole UI, load our code, ‘insert’ in a div
● drive the viewer via a Javascript API
● in the ‘div’ our code has full-control so we have to respond to events and so on.
Js-Interop
Thanks guys … JSInterop rocks !!!
@JsType
public interface ViewerInteropInterface {public void load(String caseId, ...);void doThis();void doThat();
}
public class Viewer implements ViewerInteropInterface {
@JsExport("$wnd.my.viewer")public Viewer(String containerId) {}
}
Js-Interop
it’s just plain old js
<!doctype html><html><head><script type="text/javascript" language="javascript" src="viewer/viewer.nocache.js"></script><script type="text/javascript"> window.addEventListener('viewerReady', function() { var viewer = new window.my.viewer('panel'); viewer.load(....); } );</script></head>
<body> <div id="panel"></div></body>
</html>
Js-Interop
we use the std linker (and some permutation sometime)
public class Caster implements EntryPoint {
@Override public void onModuleLoad() { castReadyEvent(); }
private static final native void castReadyEvent() /*-{$wnd.dispatchEvent(
new CustomEvent('viewerReady')); }-*/;
}
Js-Interop
I’m lazy
SDM in eclipse
EntryPoint in separate module
Viewer ev = new Viewer("panel");
ev.load(...);
Data Layout from serverside in Javato Clientside … in Java too ;-)
Data Layout
● integrate with the serverside flow and provide a data outlet to be consumed by the viewer
● our response here is obviously Java (so we
write only once the code parsing the data layout)
Data Layout
● you can’t reinvent the wheel and (mesh) compression algorithms are a “consolidated wheel”
● So, we took for the low level compression algorithms suitable (highily optimized) native implementations and we plugged (with a common api) in the code java and javascript ‘side’.
Computation client side: Web Workers
Web Workers
Web Workers provide a simple means for web content to run scripts in background threads. The worker thread can perform tasks without interfering with the user interface. Once created, a worker can send messages to the JavaScript code that created it by posting messages to an event handler specified by that code (and vice versa.)
Web Workers
var worker = new Worker(scriptUrl);
and then onMessage/postMessage
Security Madness (SDM)
External scripts sucks
Web Workers
private static final native JavaScriptObject injectWorker(String code) /*-{ return (function(window) { function WorkerFactory() { var blob = new Blob([ code ], { type: 'application/javascript' } );
this.createAndStart = function(onerror,onmessage) { var blobURL = URL.createObjectURL( blob ); var worker = new Worker( blobURL ); worker.addEventListener('error',onerror,false); worker.addEventListener('message',onmessage,false); URL.revokeObjectURL( blobURL ); return worker;};
} return new WorkerFactory(); })(window);}-*/;
Web Workers
Another Story
● another story, creating workers with GWT
● gwt-ns, webworker linker
Does not works nicely with SDM, unfortunately :(
