Sanjeewa Malalgoda

Scalable Deployment Patterns in

API Manager

Tuesday, February 11, 2014

Senior Software Engineer

Uvindra Dias Jayasinha Senior Software Engineer

**

About the Presenter

๏ Sanjeewa joined WSO2 in September 2010. He is senior software engineer, focusing on WSO2 API Manager. He is core contributor for API Manager, tenant aware load balancer and stratos.

๏ Uvindra joined WSO2 in November 2013. He is a Senior Software Engineer in the WSO2 API Manager Team.

**

About WSO2๏ Global enterprise, founded in 2005 by

acknowledged leaders in XML, web services technologies, standards and open source

๏ Provides only open source platform-as-a-service for private, public and hybrid cloud deployments

๏ All WSO2 products are 100% open source and released under the Apache License Version 2.0.

๏ Is an Active Member of OASIS, Cloud Security Alliance, OSGi Alliance, AMQP Working Group, OpenID Foundation and W3C.

๏ Driven by Innovation

๏ Launched first open source API Management solution in 2012

๏ Launched App Factory in 2Q 2013

๏ Launched Enterprise Store and first open source Mobile solution in 4Q 2013

๏

*

What WSO2 delivers

**

Deployment Patterns

○ Understanding key components and data model.○ Single-server deployment○ Distributed deployment.

How to deploy across zones.Message flows of sample deployment.

○ Integration with workflow engine.○ Enabling statistics for API Manager.○ Multi data center deployment.○ Deployment with online/offline monitoring.○ How to scale up and scale out.○ API Façade deployment pattern.○ Amazon EC2 deployment.○ Fine tune

**

Understanding Key Components

API StoreThis component provides a space for consumers to self-register, discover API functionality, subscribe to APIs, evaluate them, and interact with API publishers.

API Key Manager ServerThis component is responsible for all security and key-related operations. When an API call is sent to the Gateway, it calls the Key Manager server and verifies the validity of the token provided with the API call.

**

Understanding Key Components

API GatewayThis component is responsible for securing, protecting, managing, and scaling API calls. The API gateway is a simple API proxy that intercepts API requests and applies policies such as throttling and security checks.

API PublisherThis component enables API providers to easily create, publish their APIs, share documentation, provision API keys, and gather feedback on API features, quality, and usage.

**

Data Storages

User Manager Database Stores information related to users and user roles. This information is shared among the Key Manager Server, Store, and Publisher.

API Manager Database Stores information related to the APIs along with the API subscription details.

Registry Database Used to store API metadata and other information. Shares information between the Publisher and Store.

Default carbon database embedded with API Manager binary distribution will contain all tables necessary for registry and user management.

**

Single-server deployment

PublisherStoreKey ManagerKey Manager

WSO2CARBON_DB WSO2AM_DB

94439763

82808243

**

Configurations

○ To edit API manager related configurations you need to edit api-manager.xml file located at /repository/conf directory of product distribution. To learn about configurations refer this document.

○ To configure data sources and database related configurations you need to edit master-datasources.xml file available at /repository/conf/datasources directory.01. WSO2_CARBON_DB, 02. WSO2AM_DB, 03. WSO2AM_STATS_DB

○ Configurations related to user management are available at user-mgt.xml file in /repository/conf directory(add, edit user store etc.).

○ Configurations related to registry are available at registry.xml file in /repository/conf directory(add, edit user store etc.).

**

Distributed Deployment

**

Zonal Deployment

Deployment Across zones

DMZ

**

Deployment Pattern #2 - Data Flow

API Creation

Publisher +Store

Publisher +Store

Gateway Manager

Gateway Worker

Gateway Worker

BAM Key Manager

Key Manager

Internal ELB

External ELB

**

Deployment Pattern #2 - Data Flow

API Invocation

Gateway Worker

Gateway Worker

Gateway Manager

Publisher +Store

Publisher +Store

External ELB

Internal ELB

Key Manager

Key Manager

BAM

**Login to publisher/ store

Publisher +Store

Publisher +Store

Gateway Worker

Gateway Worker

Gateway Manager

External ELB

Internal ELB

Key Manager

Key Manager

BAM

Deployment Pattern #2 - Data Flow

**Statistics collection and presentation

Gateway Manager

Gateway Worker

Gateway Worker

Publisher +Store

Publisher +Store

External ELB

Internal ELB

Key Manager

Key Manager

BAM

Deployment Pattern #2 - Data Flow

**

Partially Distributed Deployment

*

Partially Distributed DeploymentMinimum fully distributed API Manager deployment will have 1 gateway, 1 key manager, 1 store and 1 publisher(all together 4 nodes). But when you need high availability for each server type we might need 2 or more instance per one server type.

2 nodes for combined API store/publisher(api store and publisher should run on same instance). These 2 nodes should be clustered for single cluster domain. And load balancer should route all requests comes with /store or /publisher contexts to these nodes(9443 and 9763).

2 nodes for combined API gateway/key manager(API gateway and /key manager should run on same instance).

**

Pattern #3 - Workflow Integration

**

Pattern #4 - Enabling Statistics

**

Pattern #5 - HA for Statistics

**

Pattern #6 - HA for Statistics - Multiple Data Centers

**

Pattern #7 - Online/Offline Statistics

**

Scaling

○ Most of the time we need to scale gateway and key manager as it hits large number of requests

○ To scale horizontally means to add more nodes to a system, such as adding a new node to a gateway cluster.

○ To scale vertically means to add resources to a single node in a system, typically involving the addition of more computing power.

**

Scale Horizontally

○ To scale horizontally means to add more nodes to a system, such as adding a new node to a gateway cluster.

○ If we take gateway as an example, we can add more nodes to gateway cluster on demand.

○ Then load balancer should be detect newly added nodes and route requests to them.

○ This is the most common and widely used pattern of scaling API Manager deployments .

○ We can apply this pattern to gateway, key manager, store and publisher.

**

Scale Horizontally

Scale out

500 requests 500 X 4 requests

**

Scale Vertically

○ Scale vertically means add more resources to a single node in a system, typically involving the addition of more computing power, RAM and etc.

○ Implementation of WSO2 API Façade facilitate by using WSO2 API Manager to build Demand and Façade layer, WSO2 ESB to build mediation layer and connect to the existing services.

○ In API Manager we can add mediation logic, service chaining, message formatting to API definition.

○ With API Façade pattern mediation, transformation logic will handled by ESB layer and it will add additional processing power to API management layer.

**

Scale Vertically

Scale up

API authenticationMessage mediationService chaining

Message mediationService chaining

API authentication

Processing power 3 unit per request

Processing power 1 unit per request

Processing power 2 unit per request

**

Pattern #8 - API Façade pattern

**

Deploying on Amazon EC2

Using pre-built AMI

○ Switch to the US East (N. Virginia) region Use AMI ID: ami-db182fb2.○ An instance with 1.7 Gig of memory is enough for testing purposes.

Otherwise use at least a Medium size instance.○ Create a security group opening ports 22 (SSH), 9443, 9763 (tcp, API

Mgr admin console), 8280 (tcp, API GW ports).○ Start an instance and attach security group to it.

**

Deploying on Amazon EC2

Standard deployment with binary distribution

M1-Medium instance to run one Carbon instance. Note : based on the I/O performance of instance, it is recommended to run multiple instances in a Large instance (m1.large or m1.xlarge).

Deploy API manager on EC2 instances. If you have enough resources you can run multiple API manager instances with port offset.

If you need to create DMZ for gateway and store there are several options for that. One method is create VPC to isolate deployment also you can create security group and apply same security group to all nodes in same zone(here DMZ).

**

Tune up Enable response caching.

Enable caching at API Gateway or Key Manager Server (repository/conf/api-manager.xml).<EnableGatewayKeyCache>false</EnableGatewayKeyCache><EnableKeyMgtValidationInfoCache>true</EnableKeyMgtValidationInfoCache>

Enable JWT caching(repository/conf/api-manager.xml).<EnableJWTCache>true</EnableJWTCache>

Enable oauth key caching(repository/conf/identity.xml).<EnableOAuthCache>true</EnableOAuthCache>

For store UI improvement we can enable recently added API cache and tag cache.

Memory allocated for the server increase by modifying /bin/wso2server.shDefault setting : -Xms256m -Xmx512m -XX:MaxPermSize=256m. New setting : -Xms2048m -Xmx2048m -XX:MaxPermSize=1024m

**

NHTTP and PT HTTP transports(If nhttp /repository/conf/nhttp.properties). http.socket.timeout=60000 snd_t_core=200 snd_t_max=250 snd_io_threads=16 lst_t_core=200 lst_t_max=250 lst_io_threads=16

Axis2Client.xml <parameter name="defaultMaxConnPerHost">1000</parameter> <parameter name="maxTotalConnections">30000</parameter>

Set open files limit to 200000 by editing /etc/sysctl.conf(in linux) sudo sysctl -p

Tune up

**

masterdatasource.properties <maxActive>250</maxActive> <testOnBorrow>false</testOnBorrow> <validationInterval>120000</validationInterval> Mysql maximum connections mysql> show variables like "max_connections"; max_connections was 151 set to global max_connections = 250;

repository/conf/tomcat/catalina-server.xmlmaxThreads="750" minSpareThreads="150" disableUploadTimeout="false" enableLookups="false" connectionUploadTimeout="120000" maxKeepAliveRequests="600" acceptCount="600"

Tune up

**

Call to action page

๏ Product documentation home page(http://docs.wso2.org/display/AM170/WSO2+API+Manager+Documentation)

๏ Article on fine tuning API Manager(http://sanjeewamalalgoda.blogspot.com/2014/01/wso2-carbon-420-based-products-general.html)

๏ Product home page(http://wso2.com/products/api-manager/)

**

Business Model

**

Q & A

Contact us !