drupal amazon integration

Hosting Drupal – Getting Started!

Drupal Integration with Amazon S3 & Cloudfront

Mark Schoonover – [email protected] 2010

Content Delivery Networks

● What do they do?

http://www.thetajoin.com

Creative Commons Attribution ShareAlike 3.0

http://www.thetajoin.com/

http://en.wikipedia.org/wiki/Creative_Commons

http://creativecommons.org/licenses/by-sa/3.0/



● Content Delivery Networks (CDN) are used to move bandwidth intensive files off your Drupal hosting server and put them closer to end user computers.

● Content can be audio, video, photos, software, other documents, or static files.










● Excellent not just for distributing large files, but frequently accessed smaller files.










● Excellent not just for distributing large files, but frequently accessed smaller files.

● Excellent for streaming audio and video, especially if your Drupal server doesn't support streaming.







● Popular CDNs

– Akamai– Amazon Cloudfront– Amazon Simple Storage Service (S3)– CacheFly– Many more...




We'll be concentrating on integrating Drupal 6 with Amazon Simple Storage Service – S3 & Amazon Cloudfront







Requirements

http://www.thetajoin.comCreative Commons Attribution ShareAlike 3.0








Requirements

– Amazon Web Services Account– Signup for Amazon S3 & Cloudfront– Amazon S3 file management tools– Comfortable installing & configuring Drupal modules– Comfortable modifying Drupal settings.php file– Comfortable modifying template.php theme file– Know how to configure ImageCache







Benefits of Amazon S3







Benefits of Amazon S3– Unlimited, pay as you go storage– Geographical data centers in US, Northern CA, Europe

too– Many S3 file managment tools– Keeps your Drupal files directory free of large infrequently

changed files– Supports BitTorrent







Drawbacks of Amazon S3– Not as many datacenters as Cloudfront– No CCK for S3– Manual integration with Drupal content– Data in two places







Benefits of Amazon Cloudfront







Benefits of Amazon Cloudfront– Unlimited temporary storage– Uses Amazon S3 as origin storage– More datacenters than S3– Now supports audio & video content streaming







Benefits of Amazon Cloudfront– Unlimited temporary storage– Uses Amazon S3 as origin storage– More datacenters than S3– Now supports audio & video content streaming

● Unlimited, Private YouTube anyone??Unlimited, Private YouTube anyone??







United States

* Ashburn, VA

* Dallas/Fort Worth, TX

* Los Angeles, CA

* Miami, FL

* Newark, NJ

* Palo Alto, CA

* Seattle, WA

* St. Louis, MO



Europe

* Amsterdam

* Dublin

* Frankfurt

* London

Asia

* Hong Kong

* Tokyo





Drawbacks of Amazon Cloudfront– 24 hour limit on files at edge, must recopy from S3– Charged for every time file is copied from S3 to

Cloudfront.– Charged when file requested from Cloudfront– Revisions of files with same name







Copy Content to Amazon S3







Copy Content to Amazon S3– Amazon S3 Drupal module

● Can be found: http://drupal.org/project/amazon_s3– Firefox S3Fox add-on

● Can be found: http://www.s3fox.net/– There are others...







Sign Up for Amazon Web Services

– http://aws.amazon.com/







Sign Up for Amazon S3 & Cloudfront







Amazon S3 Access Keys







Install & Enable Amazon S3 Module







Install & Enable Amazon S3 Module

– Download– Untar into /path/to/drupal/sites/all/modules– Enable

● Administer->Site Building->Modules







Amazon S3 Drupal Module Configuration








Administer>Site Configuration>Amazon S3 Configuration







Amazon S3 Drupal Module Usage







Amazon S3 Drupal Module– Draw backs

● Can't create subdirectories● Very basic S3 control● Can't modify existing buckets● Can't create Cloudfront distributions







Toplevel bucket name needs to be unique across ALL of S3!







Toplevel bucket name needs to be unique across ALL of S3!

– cdn.example.com







S3Fox – Firefox Addon







S3Fox – Firefox Addon– www.s3fox.net







S3Fox Account Manager







S3Fox Configuration







S3Fox Usage







Files are now on Amazon S3!









Drupal & Amazon S3 Integration







Drupal & Amazon S3 Integration● Obtain URL to file on S3

– Copy URL to Clipboard● Insert URL into Content





Amazon S3 File URL



http://cdn.thetajoin.com.s3.amazonaws.com/HostingGuide/CactiLogin01.png





Paste S3 URL









Drupal & Amazon S3 Integration Complete!







Oh, this URL

http://cdn.thetajoin.com.s3.amazonaws.com/HostingGuide/....







Oh, this URL

http://cdn.thetajoin.com.s3.amazonaws.com/HostingGuide/....

Oh SO ugly!







DNS CNAME trick







New URL:

http://cdn.thetajoin.com/HostingGuide/CactiLogin01.png







Amazon Cloudfront Integration







Amazon Cloudfront Integration● Upload files to S3● Create Cloudfront distribution

– Right click on bucket->manage distribution







– Right click on bucket->manage distribution● Add CNAME like cf.example.com in DNS






– cf.example.com






– cf.example.com– http://cf.example.com/directory/file.pdf






– cf.example.com– http://cf.example.com/bucket/file.pdf

● Amazon Cloudfront handles requesting content from S3





Amazon Cloudfront Integration● Determine URL● Copy N Paste URL into Drupal







Amazon Cloudfront Integration● Determine URL● Copy N Paste URL into Drupal● You're golden!







Drupal and Amazon Cloudfront Automation● Use Cloudfront module







Drupal Amazon Cloudfront Module● Install Cloudfront module, enable, permissions

– http://drupal.org/project/cloudfront– Depends on ImageCache & ImageAPI modules

● Add Access & Secret Keys to settings.php● Add 'Send to Cloudfront' Image Cache action &

Configure● Override theme_imagecache function in template.php● Cron configuration & security







Drupal Amazon Cloudfront Module● Add Access & Secret Keys to settings.php







Cloudfront Config







Override theme_imagecache function in template.php







How does it work?● imagecache inserts path data into cloudfront_files

table








table● CRON job http://localhost/cloudfront/processqueue

– Pushes images to S3– Updates cloudfront_send_queue table with updated

path info







table● CRON job http://localhost/cloudfront/processqueue

– Pushes images to S3– Updates cloudfront_send_queue table with updated

path info● garland_imagecache() rewrites IMG tag URL to

point to Cloudfront








Becareful flushing imagecache!

– Could take significant time to reprocess images, copy to S3!






Becareful flushing imagecache!– Could take significant time to reprocess images,

copy to S3!– Remember Cloudfront and revisions, there's no

way to manually flush Cloudfront


Slides of this talk available on: http://www.thetajoin.com - SandCAMP

Thank you for attending my SandCAMP 2010 session. I hope you enjoyed the session as much as I did preparing and presenting.

For attendees of my sessions & people who download my slides, I'm offering a free Drupal performance analysis of your site! I have to limit this offer to a single site only, so pick your worst! Just email or call to setup an appointment.

Hope you had a great time at SandCAMP 2010!

Regards,Mark Schoonover

http://www.thetajoin.com - The Drupal Hosting & Performance CompanyEmail: [email protected] :: Voice: 619-928-4473 :: Fax: 619-374-3130

drupal amazon integration

Documents