Who Am I ?By day:

Technical Program Manager • National Rural Electric Cooperative Association • Large 40M+ Angular, Backbone.JS, .Net WebApi • application

By night

JavaScript Extraordinaire Co-Founder & Chief Engineer for Hoozip.com • Angular • AWS S3 • AWS Lambda • AWS Api Gateway • AWS EC2 @javascriptbully

• Mongo • Express • Angular • NodeJS

• AWS • NodeJS • Angular • Lambda

@javascriptbully“Time to get Mean!”

?

Who Am I ?

Time to get A.N.A.L !!!!!

Event-Driven Compute

• As data enters your cloud infrastructure Lambda makes it extremely easy to transform that data by applying your function to the data based on events you define.

• Examples include data transformation, image transformation, large scale data analysis. For example, run transformation functions whenever a DynamoDB table is loaded.

• Ability to scale horizontally across large scale work loads and dynamically then instantly shrink back once the work is complete.

• 100 concurrent executions

• 1,000 invokes per second

Quickly build mobile Backends

• AWS mobile SDKs

Lambda offering overview

• Runs code in response to events

• Object uploads to Amazon S3

• Updates to Amazon DynamoDB via Streams

• Data in Amazon Kinesis Streams

• In app activity

• Lambda handles all capacity, scaling, patching and infrastructure administration

• Lambda only runs your code when triggered so you don’t have to pay for unused capacity

• Provides realtime metrics and logs to Amazon CloudWatch

• Low cost, No upfront costs

• Code is charged in measurements of 100ms increments

• Python, Java, NodeJs

• Lambda Function (Micro service architecture)

• Prebuilt examples

• Built in support for AWS SDK

• AWS Free Tier you can try Lambda for free

• Security is baked in

Lambda offering overview (cont’d)

AWS Lambda supports the following runtime versions

• Node.js: v0.10.36 • Java: Java 8 • Python: Python 2.7

If you author your Lambda function code in Node.js, the following libraries are available in the AWS Lambda execution environment so you don't need to include them:

• ImageMagick: Installed with default settings. For versioning information, see imagemagick nodejs wrapper and ImageMagick native binary (search for "ImageMagick").

• AWS SDK: AWS SDK for JavaScript version 2.2.32

If you author your Lambda function code in Python, the following libraries are available in the AWS Lambda execution environment so you don't need to include them:

• AWS SDK for Python (Boto 3) version 1.2.3

There are no additional libraries available for Java.

Hoozip Use Case

• Want to send property reports on demand via SMS

• Wanted to follow an event driven, micro service design

• Scale our systems up or down when certain load metrics are hit

• Send out notifications to owners of properties when their property is requested

• Managing and reacting to all those events would require a complex infrastructure, so often we simply put them together in one controller action or use observers that run in the same processes as our applications. This makes the codebase more complex as parts start getting interwoven.

• Most teams start pushing those tasks into background workers, but the infrastructure necessary for managing tasks this way is overhead too. Therefore, background workers are typically limited to the most important tasks. This is especially true when they don’t get automatically triggered by events, but need to be triggered through the codebase. Doing this adds another level of complexity to the code in order to understand which part triggers which event.

• It lets you write small NodeJS functions that will be called with the event metadata from events triggered by various services or through your own code.

Hoozip

• Hoozip.com is a marketplace and software provider for real estate investors (rehabbers, wholesalers, lenders) to find, post, and curate real estate investment deals in an efficient way.

What we normally would do

• Size Provision Scale Servers

• Estimate capacity

• Run fault tolerance

• Manage Operating System Updates

• Apply Security Patches

• Monitor for performance and availability

• Lambda does the impedance matching for your event flow so you don’t have to worry about over or under provisioning as Lambda scales elastically

call twilio post to ApiGateway

call Lambda

parse messagefuzzy addressgeoCode addresscall property APIcall bitly APIpersist to DynamoDBgenerate message

post back to twiliosend message

How we did it…

* As the API get’s busier, we’ll add a queueing / push notification mechanism via SQS & SNS

TEXT YOUR DEAL ADDRESS

TO HOOZIP!(202) 798-1333

Hoozip Architecture

• We serve our Angular app out of a S3 Bucket

• We use CloudFront as our CDN for all static assets including the Angular App

• Most of our services are separate stand alone projects and handled via Api Gateway and Lambda

• We currently use DynamoDB for persisting and Postgres as our store for financial related actions and some GeoSpatial calculations.

• When we can’t use Lambda (static IP addressing, computationally heavy work loads) we use EC2 with Node v4.0, Express, served via Node Forever and NodeMon

LET’S CODE!!!!enough of that

What’s available while programming in Node?

• Lambda Function Handler

• Event Object

• Context Object

• ImageMagick available (Node.js)

• Logging & Monitoring (CloudWatch & CloudTrail)

• Exceptions

• Ephemeral I/O operations while function is executing

• Awesome! But the worst part:

• Node v0.10.36

• http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

Code via the online code editor, or in your own IDE

• Let’s build a simple API that goes out to Github, and get’s a user object. • Here’s our Lambda function:

Testing your Lambda function locally

• Easy to test your logic locally

• Easy to turn this into an express module that you can then use to serve via an api somewhere else so you’re not bound to Lambda if you decide to move at a later date.

Upload your code

• Compress the items in you source folder.

• Upload the zip file directly to your Lamda function via the console.

• Better if you use a task manager like Gulp that will zip the file and upload it for you via the AWS SDK publish API

• I prefer to use Gulp to zip my code, then upload to S3 so I have artifact versioning as well. I wrote another Lambda functions that listens for upload events on that bucket then imports the code into my Lambda function.

Now let’s create an API in API Gatewaythat integrates with our Lambda Function

Let’s setup our API to have a query string parametercalled ‘name’ that we pass into our Lambda Function

Next we map the data retrieved byApi Gatewayto the Event object passed into our Lambda Function

via the mapping template

Let’s deploy our Api to a stage called ‘prod’!Staging allows you to create various versions of your

Api for testing, different clients, etc…

And we’re done!Instantly scalable to millions of users!

Here’s my Lambda function that deploys other Lambda Functions

• Event source for this Lambda function in my S3 bucket where I upload my Lambda zip files

• Uses the AWS SDK which is already loaded into memory by AWS.

Pricing Details• No hourly, daily, monthly minimums • No per device fees • Compute time is purchased in 100ms increments • 1M requests and 400,000 GB-s of compute every month for free • You only pay when you’re doing work, not when you’re idle.

Requests

You are charged for the total number of requests across all your functions. Lambda counts a request each time it starts executing in response to an event notification or invoke call, including test invokes from the console.

• First 1 million requests per month are free • $0.20 per 1 million requests thereafter ($0.0000002 per request)

Duration

Duration is calculated from the time your code begins executing until it returns or otherwise terminates, rounded up to the nearest 100ms. The price depends on the amount of memory you allocate to your function. You are charged $0.00001667 for every GB-second used.

Free Tier

The Lambda free tier includes 1M free requests per month and 400,000 GB-seconds of compute time per month. The memory size you choose for your Lambda functions determines how long they can run in the free tier. The Lambda free tier does not automatically expire at the end of your 12 month AWS Free Tier term, but is available to both existing and new AWS customers indefinitely.

Monthly compute charges

The monthly compute price is $0.00001667 per GB-s and the free tier provides 400,000 GB-s.

Total compute (seconds) = 30M * (0.2sec) = 6,000,000 seconds

Total compute (GB-s) = 6,000,000 * 128MB/1024 = 750,000 GB-s

Total Compute – Free tier compute = Monthly billable compute seconds

750,000 GB-s – 400,000 free tier GB-s = 350,000 GB-s

Monthly compute charges = 350,000 * $0.00001667 = $5.83

Monthly request charges

The monthly request price is $0.20 per 1 million requests and the free tier provides 1M requests per month.

Total requests – Free tier request = Monthly billable requests

30M requests – 1M free tier requests = 29M Monthly billable requests

Monthly request charges = 29M * $0.2/M = $5.80

Total compute charges

Total charges = Compute charges + Request charges = $5.83 + $5.80 = $11.63 per month

Pricing Example

If you allocated 128MB of memory to your function, executed it 30 million times in one month, and it ran for 200ms each time, your charges would be calculated as follows:

What should you do next?

• Take just one API and turn it into a Lambda function.

• Build a mobile app using the Mobile SDK using Lambda as your backend

• Great example of scaling a total of 57 lines of code including error handling to 16 million posts a day.

• Scrub to 26:00 to see the Zillow presentation

https://www.youtube.com/watch?v=ygHGPnAd0Uo

Zillow Use Case (cont’d)

Zillow Use Case (cont’d)

Zillow Use Case (cont’d)

Calling Lambda Functions (extra)

Call from mobile or web apps - Wait for a response or send an event and continue - AWS SDK, AWS Mobile, Rest API, CLI

Send events from S3, or SNS - One event per Lambda invocation, 3 attempts

Process DynamoDB changes records as events: - Ordered model with multiple records per event - Unlimited retries (until data expires) or determined by your code

Writing Lambda Functions (extra)

• AWS SDK by default • Imagemagick for NodeJS Lambdas • Lambda handles the inbound traffic (no web server) • Scheduled Functions! (uses cron syntax) • Allows versioning via a publish API so you can create immutable contracts with your

clients, and you can version via S3 • (Versioning happens via the ARN format FunctionName:ARN or FunctionName:

$LATEST) • You can alias the ARN, allows for each rollbacks by quickly associating an alias with a

different ARN. You can also version via the API in API Gateway by assigning the resource to a different Lambda Function.

Stateless • Use s3, DynamoDB, or any internet based storage to persist data. • Call other API’s from your Lambda function, or other Lambdas. • Uses processes, threads, sockets, i/o for file writing (ephemeral) etc… • You can add your own libraries, no restrictions

Send events from S3, or SNS - One event per Lambda invocation, 3 attempts

Process DynamoDB changes records as events: - Ordered model with multiple records per event - Unlimited retries (until data expires) or determined by your code