(cmp407) lambda as cron: scheduling invocations in aws lambda

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Guy Davies, Sophos Ltd.

October 2015

CMP407

Lambda as CronScheduling Invocations in AWS Lambda

About me

• Guy Davies, Senior Systems Engineer, Sophos Ltd

Sophos use AWS extensively in our operations to support our anti-

malware, anti-spam, and threat-detection software and hardware

devices.

Also provide security products for AWS customers to complement their

cloud security profiles:

• UTM [ free trial available! ]

• Secure Server for Linux [ AMI available in Marketplace ]

www.sophos.com/aws

What to expect from the session

• How to schedule tasks in AWS Lambda

• Overview of the various options available

• Building up a pure Lambda scheduling infrastructure

• Resources and templates to implement this yourself

Events? Scheduling?

Why would I want to schedule Lambda?

• Lambda designed as event-driven computing

• Event-driven computing is awesome

• Most of the time you want to trigger because something

happens…

HOWEVER

Why would I want to schedule Lambda?

Sometimes you just plain need to do something on a

schedule.

Examples:

• Log cleanup

• Batching up statistics

• Alarm clock

• Infrastructure automation

Scheduling options on traditional infrastructure

Unix

• cron: recurring tasks

*/2 * * * * do_something

• at: run a task at a specific timeat 1615 oct 7 <<< “mail –s’AWS talk!’ guy.davies@sophos.com”

Windows

• Scheduled tasks

• AT

Options for scheduling Lambda functions (1)

1. Spin up an Amazon EC2 instance, and use crontab to

invoke Lambda

• Why bother running Lambda at all?

• Many folks want a pure-Lambda deployment

• Running an instance means more to manage

• Not hugely financially efficient

Options for scheduling Lambda functions (2)

2. Unreliable Town Clock (townclock.io)

• Awesome public Amazon SNS topic

• Chimes every fifteen minutes

• Community supported

• Has a 15 minute granularity

Options for scheduling Lambda functions (3)

3. Others…

• Trigger from Amazon SWF

• Trigger from an instance in AWS Data Pipeline

• Trigger from an AWS CloudTrail upload into an Amazon S3

bucket

All of these could be your solution! But what if we want a

“pure” Lambda implementation that’s managed by AWS?

A pure Lambda scheduler

How do we generate a timing signal in AWS?

0

0.2

0.4

0.6

0.8

1

Photo: ‘Signetics NE5555N, …’ by Stefan506 is

licenced by CC BY-SA 3.0. Source:

https://en.wikipedia.org/wiki/555_timer_IC

Amazon

CloudWatch

CloudWatch as a time signal

1. Set an alarm on a CloudWatch metric

2. Alarm goes into ALARM state:

• Triggers Amazon SNS which triggers Lambda

• Lambda inverts the state of the metric

3. Alarm goes into OK state:

• Triggers SNS which triggers Lambda

• Lambda inverts the state of the metric

Configuring the CloudWatch alarm

Configuring the CloudWatch alarm

Once the metric is

inverted, the alarm will

trigger at the top of the

next minute:

1-minute resolution

Configuring the CloudWatch alarm

All three states trigger the

same SNS notification. The

Lambda function figures

which was the trigger and

sets the CloudWatch

metric to the opposite state

Putting it together

Lambda

cron

function

CloudWatch

metric

CloudWatch alarm

triggers

SNS topic10

Invoke further Lambda

functions (if scheduled)

The Lambda function

Lambda function

1. The “event” is the SNS notification that is sent by

CloudWatch.

2. Invert the value of the CloudWatch metric.

3. Jobs and schedule are managed as a separate JSON

file in the bundle.

4. Invoke any Lambda functions in the schedule that are

scheduled to be run this minute.

5. Done.

Main Lambda function

exports.handler = function(event, context) {

async.waterfall([

function (callback) { flip_cloudwatch(event,callback); },

read_crontab,

execute_lambdas

], function (err) {

if (err) context.fail(err);

else context.succeed(); });

};

Flipping CloudWatch

The event is an SNS message from CloudWatch:

var snsmessage = JSON.parse(event.Records[0].Sns.Message);

If it’s just gone into alarm, we want to reset to zero. Likewise if it’s just gone into

OK, reset to 1.

if (snsmessage.NewStateValue == 'ALARM') { value = 0.0 }

else if (snsmessage.NewStateValue == 'OK' || snsmessage.NewStateValue ==

'INSUFFICIENT_DATA') { value = 1.0 };

Push the new value to CloudWatch:

var params = { MetricData: [ { MetricName: 'LambdaCron', Timestamp: new Date,

Unit: 'None', Value: value } ], Namespace: 'LambdaCron' };

cloudwatch.putMetricData(params, function(err, data) { . . . });

Crontab-like Lambda configuration

{

"jobs": [ {

"schedule": "*/3 * * * *",

"function": "testfunction",

"args": {

"key1": "test1",

"key3": "test3",

"key2": "test2"

}

} ]

}

Checking the schedule (1)

Use the fantastic cron-parser library (https://github.com/harrisiirak/cron-parser/

MIT licenced)

var parser = require('cron-parser');

Create a Date object which refers to the top of the current minute to compare

the schedule to.

var d = new Date();

d.setSeconds(0);

d.setMilliseconds(0);

Parse the crontab to find the ‘next’ runtime for the job (= now if it needs to run)

var interval =

parser.parseExpression(job["schedule"],{currentDate: d});

var runtime = interval.next();

Run each job that needs running

if (datestring == runtimestring) {

var lambda = new AWS.Lambda();

var params = {

FunctionName: job["function"],

InvocationType: "Event",

Payload: JSON.stringify(job["args"])

};

lambda.invoke(params, function(err,data) {

if (err) iteratorcallback(err);

else iteratorcallback(null);

});

}

}

Demo

Use cases

Trialling this for intelligent scaling on a schedule

• AWS scheduled scaling has limitations:

• Absolute number of desired instances

• Limit of 120 tasks per month

• Lambda function triggered by lambda-cron could do for

example:

• Every morning at 08:00 UTC, add 20% of capacity unless we

have > 30 instances already running

Reliability and monitoring

Empirically pretty reliable.

Running since April without (much!) intervention.

Even comes back after outages!

Reliability and monitoring

Monitoring: CloudWatch!

• Lambda invocation metrics will tell you that it’s running

• Application-level monitoring on the jobs you are

triggering

60 invocations / hr

Summary

1. Use CloudWatch alarms as two states to provide a

timing signal to Lambda.

2. Trigger off all three states to enhance reliability.

3. Allows us to schedule tasks purely within Lambda.

4. The cron function invokes once a minute.

Resources

Github:

github.com/g-a-d/lambda-cron

Email:

guy.davies@sophos.com

AWS

CloudFormation

stack

Lambda

function

Resources

Libraries used:

• async: https://github.com/caolan/async

• avoid nested-callback-hell (great for folks more used to

procedural programming)

• MIT license

• cron-parser: https://github.com/harrisiirak/cron-parser

• parse crontabs

• MIT license

Resources

Remember to check www.sophos.com/aws

• UTM for AWS (free trial available)

• Secure server for AWS

• Free trials, free home use AV

Remember to complete

your evaluations!

Thank you!