api speedandperf ebook

ENSURING API SPEED AND PERFORMANCE: A GUIDE TO API TESTING AND MONITORING FOR THE CONNECTED WORLD 1

Ensuring API Speed & Performance:A Guide to API Testing and Monitoring for the

Connected World

https://smartbear.com

https://smartbear.com/product/ready-api/loadui/overview/



LEARN MORE ABOUT LOADUI NG PRO

LoadUI NG Pro is hands down the easiestway to run a quick API load test, either against

a single web service endpoint or based off of an existing functional API test created

in SoapUI NG.




http://smartbear.com



ContentIntoduction 4

API Load Testing: The Key to Staying Ahead of Performance Problems 8

1. Start with the new code first 11

2. Test for problems on the back end 11

3. Prove SLA for third parties 12

API Monitoring: Stay On Top of Performance and Gain the Insights You Need 13

1. Identify your critical path(s) 15

2. Determine the right frequency 15

3. Figure out where your users are 16

Quality as a Continuous Goal: Bringing Testing and Monitoring Together 17

1. Build shared expectations for API design, testing, and monitoring 17

2. Make pre-production testing and monitoring inseparable 17

3. Set your team up with the right tools 18




Speed. Performance.Reliability. These are the characteristics we strive for when de-

veloping, deploying, and maintaining a truly powerful

application. These are also the characteristics our custom-

ers will use to form their opinions about the performance of

that application.

What’s the best way to achieve this application perfor-

mance gold standard? Through testing and monitoring to

ensure that it works — and that it works well, consistently,

and that any major bugs or glitches are caught early and

repaired before the application (or any updates or releas-

es) is deployed into production.

But in today’s connected world, the performance of

your application also goes far beyond the applications

in front of the software that your teams build and man-

age. APIs are the glue that holds the connected world

together — the means by which various applications

and components become interdependent and infor-

mation is relayed from one device, application, or da-

tabase to another. This process can seem trivial at first,

since a well-performing API is virtually invisible to your

customers, as well as anyone else not directly involved

with testing or development. But the opposite is also

true – if your API is underperforming, your customers

will feel the pain. As such, ensuring the performance of

your API is a necessity when building a fast and reli-

able application.

When an API breaks or malfunctions, your application

feels the effects and so do your users. As our society

becomes increasingly fast-paced, even a minor delay

is likely to lead users to abandon your application in fa-

vor of something else, and word of problems, failures,

or availability issues can spread quickly across social

media. Your APIs are the backbone of the service(s)

your company delivers. They need to perform well in

order for your applications to work as desired.




According to SmartBear’s 2016 State of API Industry Survey

— performance is a top priority for API providers, with 74%

of providers reporting that performance is the best measure

of success for their API. More impressively, the number of

providers who cited performance as their top measure of

success outnumbered the total of providers that looked at

subscribers, revenue, or retention as their best measure of

success combined.

Not surprisingly, performance is also a major factor for API

consumers — with one-third of consumers reporting that

responsiveness and performance is a top priority when

selecting an API. The pervasiveness of this interdependent

application/API relationship is easily illustrated with a pop-

ular application – Yelp. Millions of hungry users turn to the

online listing and review application to find their neighbor-

hood’s best restaurants and other highly-rated businesses.

On its own, Yelp is a powerful platform that provides an

incredible amount of data, but Yelp also depends on third

parties to meet the expectations of its growing user base.

When a consumer searches for a local restaurant, Yelp also

pulls in third party data from the Google Maps API, which

enables consumers to get directions and find restaurants

that are located in their area.

Another example of this relationship is the popular mo-

bile application Waze. Waze uses the Google Maps API

in conjunction with a social platform to keep track of

traffic and weather conditions as well as road hazards.

Many people rely on Waze to help them beat traffic and

get from place to place as quickly as possible.

If the Google Maps API lags or goes down due to exces-

sive usage, the end user suffers and will likely abandon

the application and seek another platform to find the

information they’re looking for. This is bad news for Yelp

and Waze, as the user will attribute the poor experience

with their application, without caring that a third party ac-

tually caused the problem. Basically, users expect appli-

cations to work and work quickly, regardless of the level

of complexity or the number of third party or dependent

API calls involved in the transaction. If the application

doesn’t work, they won’t bother to look further for an

explanation, they will simply move on to another appli-

cation. In today’s highly competitive market, that means

that you could be losing valuable business due to a third

party API not meeting expectations.

If the problem persists, Yelp or Waze would likely seek

alternatives to the Google Maps API, which would be a




costly loss for Google, which generates significant revenue

from its API.

Third party APIs are just one aspect of the ever-growing

API economy that exists today. Of course, APIs like Google

Maps for directions, or Facebook or Twitter for application

logins are incredibly important, but there are also inter-

nal APIs that organizations need to monitor and protect

to maintain the integrity of internal systems. Additionally,

some APIs act as standalone services, passing values back

and forth to databases, making calls to verify credit card or

user information, or accessing back end systems.

You may be launching a new application that’s dependent

on legacy IT systems. It’s important that your legacy sys-

tems can interact with that new application in meaningful

ways. Internal APIs enable companies to connect core

systems with new applications. This gives them the flex-

ibility they need to access information or perform tasks

and also gives them the stability of connecting to a critical

internal database. But much like third party APIs, when

internal APIs fail there can also be costly results. If different

departments within your company rely on the API to share

information, perform vital functions, or even do their daily

jobs, a failure in your internal API could result in a loss of

productivity or vulnerability in the security of your data.

Also, when you consider that many public APIs begin as

internal APIs before growing into opportunities to gen-

erate revenue or build partnerships, it’s easy to see that

the same standards of performance should be set for all

APIs even if they are only being used within your orga-

nization. Performance matters, no matter how small the

API or how many (or few) dependencies it has.

How can you ensure API performance?

Just like you use tools to test and monitor your applica-

tion, you also need to invest in the right tools for testing

and monitoring your API. Whether you’re launching an

API of your own, or are concerned about the third party

APIs that power your applications, you need to under-

stand how your APIs are performing. You also need to

understand the capacity of these APIs so that you can

determine the amount of volume your applications can

handle and adjust as necessary. It could be a big prob-

lem if your API were to break or slow down beyond an

acceptable standard at a key moment, such as a sale or

promotion.

In this eBook, we will look at two of the most important

processes for ensuring the performance of your API




both during development and in production — API load/

performance testing and API monitoring.

We’ll look at the benefits of each of these API performance

strategies and introduce you to processes that you can

follow to implement them today. We’ll also show you how

to identify the right tools for implementing these processes

so that you can set your team up for powerful API success

for the full API lifecycle, from development through deploy-

ment and continuous integration.




API Load Testing: The Key to Staying Ahead of Performance Problems

Whether you’re developing an API to power other appli-

cations or building an API within your own corporate infra-

structure, you need to establish a reliable testing process

to prevent performance problems and understand any vul-

nerabilities or weaknesses in your API so you can prepare

for failure scenarios.

Being caught unaware when performance problems occur

with your APIs can be costly and — if you aren’t prepared

with the right processes in place — time consuming to fix.

The damage done to your application’s reputation in the

event of an unforeseen failure can be irreversible.

Hopefully, you’re already using functional tests to ensure

that your APIs perform as intended and that they meet your

business needs from day one. This is a critical first step, but

how can you ensure that your API will perform as expected

in a real world situation? And how can you ensure that your

SOAP and REST services can stand up to heavy traffic in high

demand situations? This is where load testing comes in.

API load testing is about creating production level load

simulations within an application or system that is as

near as possible to being a finished product ready to

deploy, or an existing application that’s already being

used by customers. By utilizing specialized testing soft-

ware, load testing allows testers to answer questions

like “Is my system doing what I expect under these

conditions?”, “How will my application respond when a

failure occurs?”, and “Is my application’s performance

good enough?”

“

”

A load test enables you to measure response times, throughput rates, and resource-utilization levels, and to identify your application’s breaking point, assuming that the breaking point occurs below the peak load condition.

- Microsoft guide Performance Testing

Guidance for Web Applications




API load testing tools allow you to manage important fac-

tors within the load test like test length, failure conditions,

and how virtual load will impact your API to create the most

realistic possible scenario. You can generate load either

via rate, to simulate machine-to-machine behavior or vir-

tual user, to simulate actual users interacting with your API

models, change the wait time between executions, and

specify which test cases you want to load test.

Fortunately, with the right tools, it is not necessary for load

testing to add a significant amount of bandwidth to your

API testing workload – certain tools like LoadUI NG Pro

will help simplify the process even further by allowing you

to reuse functional test scripts for load testing with just a

few clicks of a mouse, allowing you to seamlessly add load

testing to your existing functional testing process.

Furthermore, load testing can be automated just like func-

tional testing, so that you can run your load tests late at

night to avoid potential customer impact and testing re-

sources.

You’ll be able to use these tests to:

• Test the speed and scalability of new changes to your APIs in minutes

• Preview API performance behaviors before

releasing to production environments

• Shift performance insights more to the left so developers build more reliable code

• Stress test your API to see where and

when performance degradation occurs




are important to simulate through tooling, as opposed

to in production. You may run into a situation where

you can’t generate the volume of calls necessary for a

proper load test without inadvertently bringing down

your application. For example, if you’re contracted with

a third party that allots you 1,000 calls a day or 10,000

calls a day — you’re going to reach that limit pretty

quickly when conducting a load test in production. If

that’s the case, you can use an API Virtualization tool

(like SmartBear’s ServiceV) to virtualize the third party

call and both avoid over consumption of third party API

calls and remove metrics relating to dependent API

calls to get the purest possible data about your API’s

performance.

With API virtualization, minimal resources are required

for standing up and testing the API. You are able to

test the API itself, not the end-to-end application with

all its required backend systems. This eliminates the

need for a clone of the production environment. That

means you don’t have to reset and reload downstream

data anymore. You have a sandbox environment that

can be used to stress test your API through a variety of

different scenarios and you can measure and compare

You also have the ability the include performance asser-

tions, separate from the response assertions you can add

via your SoapUI functional test scripts, which provide a

quick way to specify tolerances and constraints that are

acceptable to you for a wide array of metrics, and to stop

the tests when such conditions occur. You can also use

response assertions added via SoapUI to drill into test

case level metrics — for example a test case should fail if

response time exceeds one second — and can focus in

on specific steps, like how an API performs in the check-

out process within your application.

Let’s say that you’re developing a mobile banking appli-

cation and you’ve set up a functional test to ensure a us-

er’s full name gets pulled in from your database through

an internal API. You have performed functional tests on

your API and as a result, you know that the typical re-

sponse time is 200 milliseconds and now you want to see

how that API will perform under a high volume of traffic.

When using LoadUI NG Pro for API load testing, you have

the ability to reuse an existing functional test script for

performance or load testing to see how increasing load

via virtual users impacts the API’s behavior. These tests




timeframe around it. That way, when conducting the

test, you’ll be getting a more reliable view of how your

API will perform in a real world situation.

Establish specific requirements and make sure the

entire team is incorporating them into their continuous

delivery cycle, or, if you haven’t achieved continuous de-

livery yet, make it a part of your pre-release testing plan.

2. Test for problems on the back end

When you’re looking at overall performance, it’s important

to understand what will happen on the front end when

back end problems arise.

If you have the right tools, this type of full view testing is

simple. With a tool like LoadUI NG Pro, you can set up

tests that replicate what will happen when a third party or

dependent API is unavailable, slow, generating errors, or

another malfunction occurs.

Maybe you’re highly reliant on another third party API

that’s a small business. You need to know what would

those results to get data about the performance of your

API that might otherwise be difficult or cumbersome to

obtain.

As you get started with load testing your API for perfor-

mance-related problems, there are a few areas you’ll

want to focus on:

1. Start with the new code first

New code can introduce a certain amount of chaos into

your systems when you put it into production. Don’t let

new code slip through untested. Catch errors, bugs,

performance problems, and get ahead of the game by

establishing specific requirements in your business cycle

to test new code before it’s put into production.

To ensure the quality of your tests, it’s important to pro-

vide specific requirements for how many business users,

or how many transactions, to use in your load test. Part of

that requirement will also need to include volume over a

period of time. For example, if you have a statement that

says “When I click this button, I’m going to send a thou-

sand emails,” you need to be specific and also provide a




others — you need to understand the impact third parties

can have on your API’s performance and how your perfor-

mance can impact others.

One of the challenges of working with APIs is that it has

been historically difficult to load test third party APIs and

services. Often, companies will establish automatic IP

restrictions to ensure a load test from outside their system

doesn’t overwhelm their APIs, potentially causing service

degradation.

It’s your responsibility to make sure that third parties can

prove their SLA, verify this claim on your own, and to

make sure they also have an established process for load

testing their API. In addition to using virtualization to test

the impact of poor performance of a third party API, you

should also ask them to share the results of their own load

tests to ensure they are properly testing their APIs.

happen if their API doesn’t meet their SLA, or Service Level

Agreement. You need to be able to simulate that possibility in

your load testing approach to see what happens to the rest

of the transactional process from a performance perspective,

when an event like that occurs. That knowledge will enable

you to better understand how your API will perform when

the API calls it depends on aren’t available or are responding

too slowly. From there, you can optimize your API to handle

situations of this nature, or ensure that the appropriate error

messaging is delivered.

This is another example of how virtualization can be used to

your advantage. When you set up a virtual API, you can con-

trol the performance behavior so that you can simulate what

happens when an API goes slow or even unresponsive in

the middle of a long-running transaction. You can use virtual-

ization to simulate any API and control its behavior, whether

it’s your API or it belongs to someone else.

3. Prove SLA for third parties

In the connected world, third parties are a fundamental part

of in the API economy. Whether you’re building APIs that

depend on other APIs to function or working with client that

integrates with a number of APIs, or your API is consumed by




You will have to worry about three areas when it comes

to monitoring your APIs:

• Availability: Is the service accessible at all time?

• Correctness: Is it responding with the cor-rect payload made up of the correct con-stituent elements?

• Performance: Is it responsive and coming back with the correct response in an ac-ceptable time frame?

API monitoring allows you to determine how your APIs

are performing and compare those results to the per-

formance expectations set for your application. Moni-

toring will enable you to collect insights that can then

be incorporated back into the load testing process.

When setting up monitors for your APIs, it’s important

to consider that performance may differ if your APIs

interact with multiple audiences — whether it’s an

internal, external, or public API. For example, internal

APIs will typically perform faster than a public solution,

where there are a lot of users coming to consume that

data. So, you’ll want to establish different standards

and SLAs for each of those groups.

API Monitoring: Stay On Top of Performance and Gain the Insights You Need

If you’ve made your API available to other developers,

either in a controlled fashion to trusted partners or in a

public way to anyone with a developer/production key,

you take on a responsibility to ensure that nothing affects

the API’s performance.

Factors like server load, the size of the payload, level

of encryption, and the quality of the controller code, all

affect API performance. So while load testing your API is

crucial for identifying potential problem areas, you should

also monitor those factors after you deploy. That’s where

API monitoring comes in.




Let’s consider the banking application example again.

Similar to when you created your load test, you can use

your original functional tests to set up monitors as well

when using Ready! API and AlertSite. If you’re using the

functional test script to test an API that’s pulling in a us-

er’s full name from your database, you can set up a per-

formance monitor for that test case.

Once you’ve created your monitors and established

your acceptable thresholds, you can set up alerts to be

notified if performance degrades or the API goes offline.

Choose the interval that you’d like to test the perfor-

mance of the API while in production. This allows you to

find problems and fix them before they impact your cus-

tomers. You can also choose a variety of locations to test

from, both inside and outside of your network. This will

allow you to test your APIs live, in production, from the

geographies where your customers are.

If you’re providing your own APIs, reusing functional API

tests created during development and testing for moni-

toring has several advantages:

• Instead of just checking availability, the actual func-

tionality of your API will be scrutinized on a set

schedule, providing you with a safety net for con-

tinuous deployment practices and infrastructure

changes.

• API monitors that use full test scripts provide far

more data than those that only monitor a single

endpoint. Being able to follow the desired workflow

of your business critical transactions will give you

far greater visibility into performance metrics than a

standard “up/down” monitor alone.

• Given that your functional API monitors mimic

expected usage scenarios, their actual structure

can tell Ops how your APIs are expected to be

used, and help them set up the API infrastructure

accordingly.

• Using one tool for creating tests and monitors re-

sults in lower overhead in maintenance, learning,

cost, and man hours.




When setting up monitors for your API, there are few

important details you’ll need to consider:

1. Identify your critical path(s)

To effectively monitor API performance, you first need

to identify the critical path for the end user. What are the

most used or traversed areas within your application?

These high trafficked areas are critical for your organiza-

tion and need to be monitored at all times.

If you’re a large company, you will typically have a provid-

er to give you the analytics data, and should understand

thoroughly what’s going on. As a smaller company, you

may need to do more of the work on your own, but still

need to ensure these areas are being watched and that

the correct APIs are being monitored for performance

and availability.

In this case, a tool like Google Analytics can be used to

track traffic within your application so that you can iden-

tify the areas that experience a high volume of activity.

If there’s an API tied to that specific area of the applica-

tion — for example, if Yelp knows that there is significant

volume going to its maps — you’ll know that’s an API that

needs to be monitored and that alerts should be set up

if performance suffers or availability is impacted.

2. Determine the right frequency

It’s important to determine the frequency that you want

to use execute tests to monitor your API. A good way

to determine that frequency is to think about the pain

tolerance within your organization — how long does it

take before someone identifies a problem?

What is the period of time before someone runs into

the room asking why the service isn’t available, or the

website doesn’t look good, or something isn’t function-

ing properly? How much of an outage would be tolerat-

ed by management? By the C-level executives?

For a typical web service request being tested for end-

point availability only, that frequency may be as little as

every 5 minutes or as much as every hour. For a multi-

ple-step transaction involving more than one request,

you may want to stretch that out to 10 to 15 minutes or

as much as every 2 hours, depending on your organi-

zations thresholds and budget.




Monitoring at consistent intervals will provide the visibility

you need into performance, without having to run load

tests in your production environment. By monitoring over

time, you’ll be able to collect data which can be used to

adjust the parameters of future tests, based on availability

and performance data over time.

3. Figure out where your users are

In addition to determining the high traffic areas within an

application, you also need to pinpoint the sources of your

traffic. Where is your traffic coming from, geographically

speaking? If customers are accessing your APIs via the

web, what browsers are they using? How are you track-

ing the performance for end users who are coming from

these sources?

Setting up monitors to track performance for these sourc-

es allows you to focus on the right areas and ensure high

quality performance for all of your traffic sources in the

areas where your customers are located and using the

technologies they employ to access your APIs.




Quality as a Continuous Goal: Bringing Testing and Monitoring Together

Testing and monitoring are both critical processes for

ensuring success for your API. But how do you actually

bring these processes together to deliver value and miti-

gate performance issues?

Below are three steps designed to point you in the right

direction and start conversations that will help you get your

performance strategy up and running.

1. Build shared expectations for API design, testing, and monitoring

It’s important to have established expectations that are

shared across your teams. This should include expecta-

tions for the design of your software but also for the testing

and monitoring of your APIs. Architects need to set shared

expectations for performance at their level, which then

propagates to developers. Then developers need to make

sure they have tools to ensure that the performance char-

acteristics match the expectations at their level. And

testers need to be aligned on those expectations as

well.

By setting expectations that can survive across groups,

you limit the risk of putting your team in a position

where you’re finding out after the fact that something

isn’t scalable because of an architecture difference or

figuring out in ops that you underestimated the hard-

ware needed for your API to be considered successful.

2. Make pre-production testing and monitoring inseparable

In the connected world, experts from the testing and

monitoring worlds need to come together to produce

valuable changes in how your organization delivers

value through its APIs. Incorporate what you’re learning

from monitoring your API’s performance into develop-

er expectations. The insights you gain from monitoring

should enable you to learn important lessons based on

past performance. This will help you determine which

goals are important, and whether or not the goals you

set in the beginning are logical based on the data, and

whether or not they are the right ones to be focused on.




Make ensuring the quality of your APIs and applications ev-

eryone’s concern. Involve testers and developers in discus-

sions regarding monitoring and testing your APIs. Ensure

that everyone is on the same page and understands their

vital role in ensuring API success within your organization.

You’ll also be able to determine things like the quality of

your validation components and what you should be using

to determine success in tests in future cycles.

3. Set your team up with the right tools

At the end of the day, achieving a high level of API perfor-

mance will require having tools that are designed with APIs

in mind. These tools should include the ability to automate

specific processes and should also integrate into the pro-

cesses you have setup within your team.

From a monitoring perspective, these tools should

also provide a perspective beyond the API into your

other areas of your application. Having broader visi-

bility will make it easier to identify the root cause of

problems when they arise — whether those problems

are happening within your API or application, or from a

third-party outside of your corporate infrastructure.

The ability to reuse test scripts, thus reducing man

hours and increasing productivity, for a variety of test-

ing types, such as functional, performance, security, and

synthetic monitoring, is another highly desirable option.

If your team has already invested the time into creating

tests, it makes sense to get the most out of those ex-

isting scripts before turning to other resources for the

missing components of your life cycle testing.




Speed is just as important as accuracy when it comes to APIs. A slow API can grind the user experience to a halt, but you don’t

have to wait for things to go live before understanding how they will behave under

heavy load.

TRY IT FOR FREE




https://smartbear.com/product/ready-api/loadui/free-trial/



Put SmartBear’s API Readiness Tools toWork for Your Team Today

APIs are critical to your business. Whether you are providingAPIs or consuming them, you need to:

Ready! API combines the power of SoapUI NG Pro, LoadUI NG Pro, Secure, ServiceV, and API Monitoring in AlertSite into a single pane of glass.

From functional testing, to performance testing to post-deployment monitoring, SmartBear’ API tools help you to deliver accurate, fast, and secure APIs.

Try it free today! Visit SmartBear.com to start your

• Visualize what they do

• Validate that they function as intended

• Virtualize them for use in agile testing

• Monitor them to make sure they’re available and performing well

FREE 14-day trial of Ready! API



https://smartbear.com/product/ready-api/soapui-ng/overview/


https://smartbear.com/product/ready-api/secure/overview/

https://smartbear.com/product/ready-api/servicev/overview/

https://smartbear.com/product/alertsite/overview/


https://smartbear.com/product/ready-api/free-trial/


Over 4 million software professionals and 25,000 organizations across 194 countries

use SmartBear tool

4M+ 25K+ 194users organizations countries

See Some Succesful Customers >>

APIREADINESS

TESTING PERFORMANCEMONITORING

CODECOLLABORATION

Functional testing throughperformance monitoring

SEE API READINESS PRODUCTS

Functional testing,performance testing and test

management

Synthetic monitoring for API,web, mobile, SaaS, and

Infrastructure

Peer code and documentationreview

SEE TESTINGPRODUCTS

SEE MONITORINGPRODUCTS

SEE COLLABORATIONPRODUCTS



http://smartbear.com/company/customers/

http://smartbear.com/product/ready-api/overview/

http://smartbear.com/products/testing/

http://smartbear.com/product/alertsite/overview/

http://smartbear.com/product/collaborator/overview/


api speedandperf ebook

Documents