load testing with redline13: or getting paid to dos your own systems

Or getting paid to DoS your own systems

Load Testing with RedLine13

Jason Lotito

What is Load Testing?

Load testing is the process of putting demand on a software system or computing device and

measuring its response.

Load testing is performed to determine a system's behavior under both normal and

anticipated peak load conditions.

It helps to identify the maximum operating capacity of an application as well as any

bottlenecks and determine which element is causing degradation.

What about Stress Testing?

Stress testing focuses on throughput. Load testing focuses on response times.

Both can be performed at the same time.

Load Testing

• Putting demand on a system

• Measuring the response

• Determine behavior under load

• Determine maximum operating capacity

• Bottlenecks

Other Load Testing Tools

• ApacheBench (ab)

• Apache JMeter

• siege

• BlazeMeter

Why RedLine13?

• Simple, up and running in little time

• Inexpensive, you pay for what you use

• Scriptable, so you have complete control over your tests

• I know Rich

RedLine13 has a bunch of features you can read about on the website. I’m really here to talk to you about the cool stuff I do with RedLine13.

The Basics of Load Testing

The Basics of RedLine13This is another subtitle that I’m apparently supposed to use, in place of the title, and the super subtitle above the title.

Preparing for Load Testing

Need• Production-level systems to test against

• Monitoring

• CPU, memory, I/O, etc.

• Application dashboards

• Logging

• Automated systems to deploy

• Dedicated people

• A time to test

Learning

• Set goals, e.g.

• 50ms

• 500rps

• Test monitoring

• Scaling

• What happens to the app under load?

• What slows down first?

As Load Tester, you are responsible for…

• Ensuring those involved with the system under test are aware and available

• Putting the system under test under load

• Completely destroying the system under load

• Providing a clear and concise write up of exactly what occurred

• Take charge! It’s fun!

Ramp Up Time

• Start at what you expect normal traffic to be

• 500rps

• Start a second test that is just the burst traffic

• 900rps

• Ramps up over 20 mins

• Lasts 40 mins at peak

• Ramps down over 20 mins

Reporting

Reporting Results: Shortest Version

• 99% @ 59ms

Reporting Results: Short Version

• Short version: During an hour starting with 0rps to 1400 rps in the first 10 minutes....

• ...when starting with 5 instances and scaling to 11 instances, the response time were: 50% 23ms, 75% 54ms, 95% 303ms, and 99% 1069ms.

• ...when starting with 11 instances, the response times were: 50% 16ms, 75% 24ms, 95% 45ms, and 99% 59ms.

Reporting Results: Detailed Version

• Provide more information

• Results

• Story driven

• Have data supporting results prepared in case

• Account for changes such as auto-scaling

• With auto-scaling: 99% at 1069ms

• Without auto-scaling: 99% at 59ms

Testing elasticsearch on c3.4xlarge

Detailed ReportingAlso included in this report was a link to the actual GitHub repository. Make sure you are keeping your load tests in version control!

Tag your load test in gitAssociate the tag with a Load Test,

so you can replay any load test easily

PHP & Node.jsWe’ll be using Node.js for our examples

Writing a RedLine13 Custom Load Test in Node.js

• Uploading a *.tar.gz/*.zip file

• npm install is run

• Expects a CustomTest.js file in root directory

• CustomTest.js should export a ‘class’

• Will create a new object from

• class(redLine, testNum, rand, config)

• Will call runTest() on object of class

{

"name": "reloaded",

"version": "1.0.0",

"description": "Recommendation Engine Load Tester",

"main": "app.js",

"scripts": {

"test": "echo \"Error: no test specified\" && exit 1",

"start": "node app.js",

"build_test": "rm reloaded.tar.gz && tar --exclude='node_modules' --exclude='.git' --exclude='logs' -cvzf reloaded.tar.gz *"

},

"author": "Jason Lotito <jlotito@meetme.com>",

"license": "ISC",

"dependencies": {

"async": "^1.3.0",

"elasticsearch": "^5.0.0",

"lodash": "^3.10.0",

"sleep": "^3.0.0",

package.json

{

"name": "reloaded",

"version": "1.0.0",

"description": "Recommendation Engine Load Tester",

"main": "app.js",

"scripts": {

"test": "echo \"Error: no test specified\" && exit 1",

"start": "node app.js",

"build_test": "rm reloaded.tar.gz && tar --exclude='node_modules' --exclude='.git' --exclude='logs' -cvzf reloaded.tar.gz *"

},

"author": "Jason Lotito <jlotito@meetme.com>",

"license": "ISC",

"dependencies": {

"async": "^1.3.0",

"elasticsearch": "^5.0.0",

"lodash": "^3.10.0",

"sleep": "^3.0.0",

package.json

{

"name": "reloaded",

"version": "1.0.0",

"description": "Recommendation Engine Load Tester",

"main": "app.js",

"scripts": {

"test": "echo \"Error: no test specified\" && exit 1",

"start": "node app.js",

"build_test": "rm reloaded.tar.gz && tar --exclude='node_modules' --exclude='.git' --exclude='logs' -cvzf reloaded.tar.gz *"

},

"author": "Jason Lotito <jlotito@meetme.com>",

"license": "ISC",

"dependencies": {

"async": "^1.3.0",

"elasticsearch": "^5.0.0",

"lodash": "^3.10.0",

"sleep": "^3.0.0",

package.json

var Test = require('./CustomTest');

var rl = require('./lib/RedLine');

(new Test(new rl.RedLine, 55)).runTest(function(){

console.log( "test complete");

});

app.js

var Test = require('./CustomTest');

var rl = require('./lib/RedLine');

(new Test(new rl.RedLine, 55)).runTest(function(){

console.log( "test complete");

});

app.js

var Test = require('./CustomTest');

var rl = require('./lib/RedLine');

(new Test(new rl.RedLine, 55)).runTest(function(){

console.log( "test complete");

});

app.js

var Test = require('./CustomTest');

var rl = require('./lib/RedLine');

(new Test(new rl.RedLine, 55)).runTest(function(){

console.log( "test complete");

});

app.js

function RedLine(){}

RedLine.prototype.recordPageTime = function(ts, time){};

RedLine.prototype.recordURLPageLoad = function(url, ts, time){};

RedLine.prototype.recordDownloadSize = function(kb){};

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

RedLine.js

function RedLine(){}

RedLine.prototype.recordPageTime = function(ts, time){};

RedLine.prototype.recordURLPageLoad = function(url, ts, time){};

RedLine.prototype.recordDownloadSize = function(kb){};

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

RedLine.js

function RedLine(){}

RedLine.prototype.recordPageTime = function(ts, time){};

RedLine.prototype.recordURLPageLoad = function(url, ts, time){};

RedLine.prototype.recordDownloadSize = function(kb){};

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

RedLine.js

function RedLine(){}

RedLine.prototype.recordPageTime = function(ts, time){};

RedLine.prototype.recordURLPageLoad = function(url, ts, time){};

RedLine.prototype.recordDownloadSize = function(kb){};

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

RedLine.js

function RedLine(){}

RedLine.prototype.recordPageTime = function(ts, time){};

RedLine.prototype.recordURLPageLoad = function(url, ts, time){};

RedLine.prototype.recordDownloadSize = function(kb){};

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

RedLine.js

function RedLine(){}

RedLine.prototype.recordPageTime = function(ts, time){};

RedLine.prototype.recordURLPageLoad = function(url, ts, time){};

RedLine.prototype.recordDownloadSize = function(kb){};

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

RedLine.js

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

Timer.prototype.start = function ()

{

return new ActiveTimer( Date.now(), this.url, this.redLine );

};

function ActiveTimer( startTime, url, redLineAPI )

{

this.endTime = 0;

RedLine.js

RedLine.prototype.recordError = function(error){console.log(error)};

RedLine.prototype.recordProgress = function(testNum, percent){};

function Timer( url, redLineAPI )

{

this.url = url;

this.redLine = redLineAPI;

}

Timer.prototype.start = function ()

{

return new ActiveTimer( Date.now(), this.url, this.redLine );

};

function ActiveTimer( startTime, url, redLineAPI )

{

this.endTime = 0;

RedLine.js

Timer.prototype.start = function ()

{

return new ActiveTimer( Date.now(), this.url, this.redLine );

};

function ActiveTimer( startTime, url, redLineAPI )

{

this.endTime = 0;

this.startTime = startTime;

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

RedLine.js

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

};

RedLine.js

var async = require( 'async' );

var sleep = require( 'sleep' );

var Timer = require( './lib/RedLine' ).Timer;

var Test = require( './lib/Test' );

// Load Testing Options

var MINUTES_TO_RUN = 60;

var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;

var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;

// Normally don't need to change

var RUN_EVERY_MILLISECONDS = 1000;

CustomTest.js

var async = require( 'async' );

var sleep = require( 'sleep' );

var Timer = require( './lib/RedLine' ).Timer;

var Test = require( './lib/Test' );

// Load Testing Options

var MINUTES_TO_RUN = 60;

var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;

var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;

// Normally don't need to change

var RUN_EVERY_MILLISECONDS = 1000;

CustomTest.js

var sleep = require( 'sleep' );

var Timer = require( './lib/RedLine' ).Timer;

var Test = require( './lib/Test' );

// Load Testing Options

var MINUTES_TO_RUN = 60;

var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;

var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;

// Normally don't need to change

var RUN_EVERY_MILLISECONDS = 1000;

function rand( low, high )

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

CustomTest.js

var sleep = require( 'sleep' );

var Timer = require( './lib/RedLine' ).Timer;

var Test = require( './lib/Test' );

// Load Testing Options

var MINUTES_TO_RUN = 60;

var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;

var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;

// Normally don't need to change

var RUN_EVERY_MILLISECONDS = 1000;

function rand( low, high )

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

CustomTest.js

var sleep = require( 'sleep' );

var Timer = require( './lib/RedLine' ).Timer;

var Test = require( './lib/Test' );

// Load Testing Options

var MINUTES_TO_RUN = 60;

var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;

var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;

// Normally don't need to change

var RUN_EVERY_MILLISECONDS = 1000;

function rand( low, high )

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

CustomTest.js

var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;

// Normally don't need to change

var RUN_EVERY_MILLISECONDS = 1000;

function rand( low, high )

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

function Reloaded( rl, testNum, rand, config )

{

this.rl = rl;

this.testNum = testNum;

this.rand = rand;

this.config = config;

this.test = new Test();

CustomTest.js

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

function Reloaded( rl, testNum, rand, config )

{

this.rl = rl;

this.testNum = testNum;

this.rand = rand;

this.config = config;

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

CustomTest.js

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

function Reloaded( rl, testNum, rand, config )

{

this.rl = rl;

this.testNum = testNum;

this.rand = rand;

this.config = config;

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

CustomTest.js

{

return Math.floor( Math.random() * (high - low + 1) + low );

}

function Reloaded( rl, testNum, rand, config )

{

this.rl = rl;

this.testNum = testNum;

this.rand = rand;

this.config = config;

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

CustomTest.js

}

function Reloaded( rl, testNum, rand, config )

{

this.rl = rl;

this.testNum = testNum;

this.rand = rand;

this.config = config;

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60, MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );

console.log( Date.now() + " - Sleeping for " + sleepInSeconds );

CustomTest.js

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60, MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );

console.log( Date.now() + " - Sleeping for " + sleepInSeconds );

sleep.sleep( sleepInSeconds );

console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );

};

Reloaded.prototype.runTest = function ( finished )

{

var timer = new Timer( 'test', this.rl );

CustomTest.js

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60, MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );

console.log( Date.now() + " - Sleeping for " + sleepInSeconds );

sleep.sleep( sleepInSeconds );

console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );

};

Reloaded.prototype.runTest = function ( finished )

{

var timer = new Timer( 'test', this.rl );

CustomTest.js

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60, MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );

console.log( Date.now() + " - Sleeping for " + sleepInSeconds );

sleep.sleep( sleepInSeconds );

console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );

};

Reloaded.prototype.runTest = function ( finished )

{

var timer = new Timer( 'test', this.rl );

CustomTest.js

function Reloaded( rl, testNum, rand, config )

{

this.rl = rl;

this.testNum = testNum;

this.rand = rand;

this.config = config;

this.test = new Test();

this.rampUp();

}

Reloaded.prototype.rampUp = function ()

{

var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60, MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );

console.log( Date.now() + " - Sleeping for " + sleepInSeconds );

sleep.sleep( sleepInSeconds );

CustomTest.js

console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );

};

Reloaded.prototype.runTest = function ( finished )

{

var timer = new Timer( 'test', this.rl );

var max = 60 * MINUTES_TO_RUN;

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

CustomTest.js

console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );

};

Reloaded.prototype.runTest = function ( finished )

{

var timer = new Timer( 'test', this.rl );

var max = 60 * MINUTES_TO_RUN;

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

CustomTest.js

console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );

};

Reloaded.prototype.runTest = function ( finished )

{

var timer = new Timer( 'test', this.rl );

var max = 60 * MINUTES_TO_RUN;

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

CustomTest.js

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

}.bind( this ), function ()

{

CustomTest.js

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

}.bind( this ), function ()

{

CustomTest.js

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

}.bind( this ), function ()

{

CustomTest.js

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

}.bind( this ), function ()

{

CustomTest.js

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

}.bind( this ), function ()

{

CustomTest.js

var max = 60 * MINUTES_TO_RUN;

async.timesSeries( max, function ( n, next )

{

var activeTimer = new timer.start();

var timeDiff = 0;

this.test.runATest( function ( err )

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

CustomTest.js

{

if ( err )

{

this.rl.recordError( err.message );

}

timeDiff = activeTimer.stop();

setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );

}.bind( this ) );

}.bind( this ), function ()

{

finished( true );

} );

};

CustomTest.js

Now, let’s hack loggingBy default,RedLine doesn’t support logging

through the web UI

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

this.redLine = redLineAPI;

this.url = url;

}

ActiveTimer.prototype.stop = function ()

{

this.endTime = Date.now();

var timeDiff = this.endTime - this.startTime;

this.redLine.recordError(timeDiff);

this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );

this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );

return timeDiff;

};

module.exports= {

RedLine: RedLine,

Timer: Timer

RedLine.js

Response Times

Counts

19 requests at 27ms

With this we can get percentilesOf course, the pro version offers this for PHP

You can use error reporting to simulate real time logging

The important thing here is…

Writing Custom Load Tests

Things to Keep In Mind

• Understand expected usage

• X% of users using the app while

• Y% are chatting with one another

• Users are logging in

• Creating accounts

• Backend systems

• Determine what’s important

Things to Keep In Mind

• User input

• Random filters

• Weighted filters

• Cached results are expected

• Client constraints

Things to Keep In Mind

• User flow through service

• Try to understand how users use the app

• Script should try to mimic

Things to Keep In Mind

• Be care about testing a single system

• System will have logging

• System will have backend services

• You’d be surprised what can cause failure

• A Load Test helps you learn before it’s in production

Things to Keep In Mind

• User interaction

• MeetMe is social, so we’ve load tested chatting

• 1 test per 2 users, both chatting with one another

• Story time!

Things to Keep In Mind

• Have developers available

• Better, have developers with you when load testing

• Find a problem, Fix it, Reroll app, Rerun test

• FFRR

• I just made that up

• Don’t use it.

Things to Keep In Mind

• Start testing from your laptop

• Seriously, my MacBook Air could bring down Erlang services

• Database indexes are a thing

• While running a load test, you can run a single client form your laptop

Things to Keep In Mind

• Someone should be testing the app/service as well

• Response times are only a number

• What does 50ms vs 300ms response times feel to the user

• What impact does 2x/3x/4x load have

• When auto-scaling, how does the client handle

Things to Keep In Mind

• Review how the client is using the API

• Review how the API developer expects the client to use the API

• Model after what the client is doing

• Call out differences early

LearnMost importantly…

In Russia, Load tests You!

Good load tests will teach you things you didn’t know about your systems.

Great load tests will help you solve those problems before they become production problems.

Load Testing with RedLine13

Thanks!@jasonlotito jasonlotito.com jasonlotito@gmail.com