drupal camp.sg 2012 session - drupal in the cloud

Drupal in the Cloud

by AzhanFounder/Director S & A Solutions

• S & A Solutions – who?• doing it with Drupal since 2007• Over 70 projects in 5 years• More than 20 clients• 99% Drupal projects• We love Drupal (obviously!)

> Drupal and S & A SolutionsDrupal in the Cloud

> Case Study 1 – http://theasiamag.com

• Content publishing of stories from Asia• Launched in 2010 using Amazon EC2+RDS• Drupal 6, on shared hosting during development• Despite performance tuning efforts, still slow pages

"Ok S & A, so what do we do now?"

Drupal in the Cloud


S & A commissioned to do a site performance evaluation

We need to•Find out what is causing the slow pages•Find out what server size is suitable•Decide between single tier or multi-tier•Provide a solution for short term/future scalability•Prove with benchmarking test results

Drupal in the Cloud


Drupal and other optimizations done:• Turned on Boost module, CSS/Javascript aggregation• Optimized images, custom module SQL queries• Removed references to missing files/images

Results:• Reduced page loading times from 8 secs to 5 secs

• average ~ 30-40% improvement• Apparent network latency stays at 3-4 secs

> Hints on bottleneck in the network, web server, database

Drupal in the Cloud


Network test:• wget download test >> 132kb/sec (good)• ping latency hovering 200-300ms

(normal for SG to USA)Network no longer the suspect

• Apache ab –k -n100 -c1 >> 2-5 secs per page load (SLOW!)• 70% content errors when ramped up to n=100, c=10

Final suspects:Web server, MySQL needs more CPU and/or memory

Drupal in the Cloud


Other opportunities in improving performance• Serve static files using a CDN (> 1GB of static files)• Load balanced web servers• Memcached / Varnish• Reverse proxy eg. Nginx• Tune & recompile Apache/MySQL for performance• Allocate more resources to Apache and MySQL (easy)

Drupal in the Cloud


TheAsiaMag asked for Amazon web services• US West Virginia availability zone

• mostly worldwide audience• 4 types of server configurations were tried• involves a Load balancer, EC2 with EBS, RDS

Drupal in the Cloud


1st run: Load balancer, 2x EC2 micro instances, small RDS2nd run: Load balancer, 1x LARGE EC2 instance, small RDS3rd run: Load balancer, 1x small EC2 instance, small RDS4th run: Load balancer, 1x medium EC2 instance, small RDS

Benchmarking tools used:• Apache ab• Drupal Devel module, MySQL slow logs• Firebug

Drupal in the Cloud


RESULTS of 1st run: Apache ab -n10000 -c10Load balancer, 2x EC2 micro instances, small RDS

EC2 CPU 100%EC2 RAM 100% (no swap)RDS CPU 100%

Apache server dies at a random time within a dayand could not revive itself

Drupal in the Cloud


RESULTS of 2nd run: Apache ab -n10000 -c10Single LARGE EC2 instance, small RDS

10,000 requests completed in under 20minsEC2 CPU 5-10%EC2 RAM 30-50%RDS CPU <5%

Impressive, but overkill. And expensive!

Drupal in the Cloud


RESULTS of 3rd run: Apache ab -n10000 -c10Single small EC2 instance, small RDS

10,000 requests completed in 1hrEC2 CPU 15-20%EC2 RAM 85%RDS CPU 50%

Performance is just OK, but cost is within budget!No need for 4th run.

Drupal in the Cloud

> Case Study 2 – http://holidaycube.com

• Property listing portal, launched in 2010• Drupal 5, dedicated server hosting• In 2011 - slow pages, occasional server deaths

“S & A, what do we do?”

Drupal in the Cloud


The problem• Just 1000+ properties• 10k page views per month• Very complex search+filters using native Drupal search• Real time statistics killing the site• Watchdog?!!!

First aid• Identified slow queries, MySQL always using too much RAM• Had to “kill” watchdog• Plenty of opportunity for code optimization, but have to wait• Hardware upgrade to rescue the site first

Drupal in the Cloud


• Current hosting at a leading US-based reputable company• Amazon web services suggested as an alternative

Drupal in the Cloud

> Scalability challenges

• Code, database, OS optimizations• Single, monster server vs multi-tier architecture• How to handle spikes in traffic/load• Security, reliability• Cost for all the fancy setup• Time to do all that?!

without breaking the bank!

> I know we could do all that with Amazon web services

Drupal in the Cloud


Proposed architecture on Amazon web services• Load balancer• 2x small EC2 for web server• 1x small EC2 for hosting static files

• Shared by all web servers via NFS• 1x LARGE RDS for MySQL database

Drupal in the Cloud


Final architecture on Amazon web services• Load balancer (later, perhaps)• Single LARGE EC2 for web server + MySQL

Keep it simple!

Building the business case for the move to AWS:• Benchmarking the AWS setup vs the current site• AWS setup has identical CPU, double RAM

Drupal in the Cloud


Amazon setup vs dedicated server

Upgrading “current” to 8GB, RHEL will cost USD700!

Current Amazon web services

CPU Dual core AMD 64bit CPU 4 ECUs (Intel Xeon dual core equivalent)

OS 64bit RHEL 4 64bit Ubuntu 11.10

RAM 4GB 7.5GB

Location USA USA (West Virginia)

Average Cost USD 300 per month USD 200 per month

Drupal in the Cloud


Benchmarking test results comparison

Apache ab requests per sec0

2

4

6

8

10

12

4

11

CurrentAWS

Benchmarking Tests

No

of

req

ues

ts p

er s

eco

nd

(mo

re i

s b

ette

r)

Drupal in the Cloud



Apache ab timing per request0

2

4

6

8

10

12

14

12

4.4

CurrentAWS

Benchmarking Tests

Tim

e in

sec

s

Drupal in the Cloud



Devel PHP timings on Homepage0

500

1000

1500

2000

2500

1941

102

CurrentAWS

Benchmarking Tests

Tim

e in

ms

Drupal in the Cloud



Devel PHP timings on Query page0

20

40

60

80

100

120

100

18

CurrentAWS

Benchmarking Tests

Tim

e in

sec

s

Drupal in the Cloud



Firebug network latency0

1

2

3

4

5

6

5.46

1.44

CurrentAWS

Benchmarking Tests

Tim

e in

sec

s

Drupal in the Cloud



Firebug full page loading time0

10

20

30

40

50

60

70

60

20

CurrentAWS

Benchmarking Tests

Tim

e in

sec

s

Drupal in the Cloud


Client is convinced!

(Yeay!!)

Drupal in the Cloud

> Why Amazon web services?

No to AWS• Could easily have been other hosting providers

• Other cloud vendors can address scalability issues• Getting over the Amazonian barriers• AWS may be viewed as “difficult” by some

Drupal in the Cloud

> Why Amazon web services?

Yes, AWS!• A platform for benchmarking, server sizing• Easy and quick to create/kill servers, run tests• Actually a safe, cheap and controlled means of

hosting your site or application• Easily spawn additional servers to cater to “high” periods

eg. A product launch lasting 4 weeksterminate the additional servers after site traffic has died down

You can get really creative designing your architecture

Drupal in the Cloud

> Drupal web load balancing

Amazon Load Balancer

Normally a complex setup with traditional hosting,the AWS Console makes it so easy to set this up in minutes

Drupal in the Cloud

> Drupal and memcached

http://drupal.org/project/memcache Amazon ElastiCache

Normally a complex setup with traditional hostingthe AWS Console makes it so easy to set this up in minutesif paired with the Drupal memcache module

Drupal in the Cloud

http://drupal.org/project/memcache

> Drupal and CDN

http://drupal.org/project/cdnhttp://drupal.org/project/cloudfront

Amazon Cloudfront or S3

Drupal in the Cloud

http://drupal.org/project/cdn

http://drupal.org/project/cloudfront

> Drupal and high availability

Amazon multi-Availability Zones for EC2 and RDS

Drupal in the Cloud

> Drupal and MySQL

Running MySQL can be done using one ofthe following options:

MySQL on Amazon EC2MySQL on Amazon RDSMulti-AZ RDS

Drupal in the Cloud

> Drupal & Amazon web services

Amazon CloudFormation“Templates” to quickly deploy a Drupal stack into the cloud

Drupal in the Cloud

> Drupal & Amazon web services

It works!

Drupal in the Cloud

> Questions?

Email: [email protected]: http://www.snasolutions.com.sg Thank you!

Drupal in the Cloud

drupal camp.sg 2012 session - drupal in the cloud

Technology

cloud drupal

cloud case study

rds drupal

drupal projects

native drupal

rds cpu case study

amazon ec2

ec2 ram