November 13, 2018

Beginner’s Guide to Cordova and Mobile Application Development

George Campbell

Lead Software Engineer

Doug Davies

Lead Software Engineer

George Campbell

Lead Software Engineer

Doug Davies

Lead Software Engineer

Agenda

1. What is a hybrid mobile app?2. How to pick a framework3. Mobile & Authentication4. Tips, Tricks and Gotchas5. Examples & Demos

• Checkout Kiosk (Doug)• Find this book by photo (George)• Scan a batch of barcodes (Karen)

What is a hybrid mobile app?

Characteristics

• Develop native apps using web technologies you are already familiar with (HTML, CSS, JavaScript).

• Leverage mobile device specific technologies via plugins: Camera, GPS, etc.

• Cross platform – Android, iOS• Code generated for Browser, XCode, and Android

Studio platforms.• One code base.

WebView

• App runs as native, but is a container for a WebView.

• Hybrid frameworks typically have plugins that bridge the WebView and native operating system so they can use native APIs.

• WebViews take advantage of GPUs on modern devices as well as fast JavaScript engines. No longer an excuse to say it’s slow.

How to pick a framework

So many to choose from!

Considerations

• Supports widest range of customer devices.

• Secure (login, datastore).

• Access to device level hardware (i.e. cameras).

• Excellent user experience.

• Rapid development cycle to support agile process.

• Easy to deploy to App Store / Google Play.

• Easy to maintain long term.

• Good community and support forums.

Why we picked Cordova

• One code base (HTML / CSS / JavaScript), many devices.

• Low cost (FREE)

• Availability of Plugins and JavaScript libraries.

• Large community with code examples and forums.

• Mature. Most problems have already been solved.

• Uses Node for a robust build environment.

• Allows rapid development in the browser.

OCLC Users prefer iOS and AndroidTypical Operating System usage by OCLC’s mobile users

One Code Base Reduces Development Cost

2 Swift Developers

(Objective C)

2 Kotlin Developers

(Android Java)

Apple Binary Android Binary

iOS User Android User

2 Web Developers(javascript)

Apple Binary Android Binary

iOS User Android User

Web App in a Hybrid Framework

Code Base inAndroid Studio

Code Base in Xcode

Maintenance!

Keepcode bases

in sync!

Mobile & Authentication

OAuth for Mobile Clients

• Typically to talk to OCLC services from your backend service you’d use a key and secret.

• However… Mobile apps shouldn’t embed the secret in the app, because it can be easily discovered and compromised.

• Thus… Mobile clients should use one of the Mobile OAuth Access Token flows.

OAuth for Mobile Clients

• Mobile app uses a web browser to open the authentication endpoint.

• User is redirected to login pages.• After a successful login the client receives back an Access

Token.• The Access Token is then used to make requests to OCLC

APIs.• The Access Token is only valid for a short period of time

and must be renewed (refresh token) periodically.

OAuth for Mobile Clients

https://www.oclc.org/developer/develop/authentication/access-tokens/user-agent-or-mobile-pattern.en.html

You can read more about OCLC’s mobile authentication patterns at:

OAuth for Mobile Clients

OCLC provides a Cordova based plugin for authentication at:

https://github.com/OCLC-Developer-Network/oclc-authentication-plugin

The plugin was developed for Digby, but we decided to open it up so anyone can use it. The Checkin Kiosk demo at the end of this presentation will show you how to use it.

We are working on a new mobile flow that will be available early next year. At that time we will be making a new Cordova plugin available and switching Digby over to use it.

Tips, Tricks and Gotchas

Don’t write it yourself!

• Most likely someone has already solved your problem.

• Lot’s of well tested JavaScript available.

• Select based on community and how active the project is. Use the forums and Slack channels!

Cordova Plugins

https://cordova.apache.org/plugins

Use the plugins!!!

Key libraries and utilities

Examples & Demos

Checkout Kiosk - Doug

https://github.com/OCLC-Developer-Network/oclc-cordova-demo

A simple self-checkout station that can run on a desktop browser or mobile device. Library staff can login and start the station for the day so that patrons can enter their user barcode and then enter item barcodes for checkout.

Uses Cordova, OCLC’s WMS NCIP service, OCLC’s OAuth2 mobile pattern, and device camera for scanning barcodes.

The fully working project will show you how to get Cordova up and running, authenticating, and making API calls.

Find this book by photo - George

Can we get an OCLC Number from a cover or spine photo?

My list of concerns when I started:

• What mobile framework will support image taking and machine learning and run on iOS and Android?

• What is the pattern for building mobile machine learning applications?

• Can I make a working demonstrator quickly with minimal code?

• How well does it work?• Will the demo spark interest by the Product Team?• How hard is it to productionize this?

Machine Learning - custom vs. canned1. Roll your own model, train it with data and deploy it.

• Start with a large, representative data set.• Build a development framework to create, train and test models.• Implement your model in your app.• Update your model if the underlying data changes.• Expensive to make and maintain.• No cost per usage but model may run slowly or have a high error rate.

2. Use a canned solution for specific use cases• Siri, OK Google and Alexa are perfect examples for voice.• Character recognition is also a RESTful http request away.• You cannot train models better than Amazon, Apple, Google and Microsoft can.• AAG & M improve their services every day with more power and accuracy.• Small cost per usage and bandwidth of sending images.• Operating system independent – perfect for hybrid.

Mobile Machine Learning - some optionsiOS• Core ML2 – fast custom Apple silicon on newest devices• “Vision” text recognition – on device requires native code plugin

• https://martinmitrevski.com/2017/10/19/text-recognition-using-vision-and-coreml/

Android• Android ML Kit – no custom silicon – relies more on cloud• “Text Recognition” – on device (sparse text) or in cloud (dense text, non-latin chars)

• https://firebase.google.com/docs/ml-kit/recognize-text

Cloud Solutions (I focused on Amazon because I’m a long time AWS user)• Amazon Machine Learning• RESTful HTTP Request – ideal for hybrid app

• https://aws.amazon.com/rekognition/

Amazon Machine Learning• Focused on API based ML solutions• You can create a free account and perform 5000 image to text translations per month

for free with Amazon Rekognition.• You can integrate your mobile app with other services, like running Restful services on

Lambdas or persisting data in S3 buckets.• Amazon, Microsoft, Google and Apple offer “wholistic” AI cloud solutions.

• https://aws.amazon.com/machine-learning/• https://azure.microsoft.com/en-us/free/machine-learning• https://cloud.google.com/products/ai/• https://developer.apple.com/machine-learning/

• “Artificial Intelligence” is actually math and statistics at scale against large data sets.• “Machine Learning” is running inputs through a trained model to get sensible outputs.

Configure Amazon Services

Find Book By Photo Architecture

www folder• Angular• JS, CSS, HTML

Cordova API• Access Camera

POST Image as a Base64 Binary File

JSON Response containing text

Amazon Rekognition

Service

OCLC WorldCat

Search API

GET with text embedded in request url

JSON Response containing MARC record

1 2

3

4

$1 per 1000 images, 5000

free per month

Demo App and Code Walk Through

Scan a batch of barcodes - Karen

Uses the camera to scan barcodes and maintains a list. The list can then be exported.

Started from Checkout Kiosk demo

Uses Cordova, OCLC’s WMS Collection Management service, OCLC’s OAuth2 mobile pattern, device camera for scanning barcodes and local storage for list of scanned barcodes.

Challenges: permissions: camera, internet, file storage

Lessons learned: can load libraries from internet rather than put them in build. Good for testing

Slack Channel

Keep the conversation going!oclcdevnet.slack.com

#oclcdevconnect

Stay connected

Doug Davies

Lead Software Engineer

George Campbell

Lead Software Engineer