scaling in mind (case study of drupal core)
DESCRIPTION
Introduce Scaling design in Drupal core, from drupal 6 to drupal 7. This slides for PHP Conference Taiwan 2014.TRANSCRIPT
Scalability in Mind當老軟體 Drupal 遇上大架構
2014-10-18 PHPConfJimmy Huang 黃雋
Drupal and me
媽，我在這裡Jimmy Major Versions
Why not Drupal?
It’s just a CMSfor damned cat not for me
Image from: https://flic.kr/p/hv9xDs
Learning Curve
Image from http://www.codem0nk3y.com/2012/04/what-bugs-me-about-modx-and-why/cms-learning-curve/
Too may reason to say no...
Not OOPNo ORM
Made by PHPHard to make theme
Hard to staging, continues deploying
沒有愛
1. FlexibilityFor Drupal Beginner
Drupal can be a:● Personal blog● Company official site● Community forum● Online commerce shopping mall● Company intranet portal● Heavy media site● Video portal● Mobile backend CMS
CMS?
Development oriented CMS● Not (only) a framework● config many things in UI● Abstract in data layer● Need 3-party modules
Content Type
Sample for phpconf 2014
Sample for phpconf 2014
View 1
View 1
View 2
View 2
View 3
View 3
Query Generator in clickssame query, different layout
Modules will working togetherhook API design
Modules will working together
/** * Implementation of hook_captcha(). */function AMAZINGMY_captcha_captcha($op, $captcha_type='') { switch ($op) { case 'list': return array('AMAZINGMY CAPTCHA'); case 'generate': if ($captcha_type == 'AMAZINGMY CAPTCHA') { $captcha = array(); $captcha['solution'] = 'AMAZINGMY'; $captcha['form']['captcha_response'] = array( '#type' => 'textfield', '#title' => t('Enter "Amazing"'), '#required' => TRUE, ); return $captcha;
module_invoke('AMAZINGMY_captcha', 'captcha', 'generate', $captcha_type_challenge);
horizontal
2. Scalability
Image from https://groups.drupal.org/node/24412
scale out
scale out
Hosting Architecturefor pure dynamic site
Prepare to scale● Reverse proxy or Hardware load balance?● Where are your Sessions?● File storage?● Separated read/write query?
Reverse Proxyip from - $_SERVER[‘HTTP_X_FORWARDED_FOR‘]
https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/ip_address/7
Reverse Proxy
Reverse Proxy
Remote addr will get proxy IPReal ip need forward from Proxy
Reverse ProxyExample setting of Nginx
Location {
proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-Host $host;proxy_set_header X-Forwarded-Server $host;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Session Storage● Plugable● Centralized ● Fast
Session Storagebefore 2008, Drupal 6 save session to DB
https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/_drupal_bootstrap/6
Session Storageafter 2011, Drupal 7, have plugable Session config in core
https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/drupal_bootstrap/7
Session Storageafter 2014, Drupal 8 include better handler from Symfony 2
Drupal 8 API: http://goo.gl/VVQ2Ua
Session StoragePHP 5.4 also have better SessionHandler class
http://php.net/manual/en/class.sessionhandler.php
File Storage● After upload
can other instance saw files?
File StorageDrupal 6 – only 1 hook nothing to help scaling
File StorageDrupal 7 – complete file handling api (hook_file_*)
File Storage● After upload, send to AWS S3 or FTP?
– Yes! by hook_file_copy● Before display, alter URL for CDN support?
– Yes! by hook_file_url_alter● When load file, streaming by other host?
– Yes! by hook_file_load
File Storage
function hook_file_url_alter(&$uri) { $cdn1 = 'http://cdn1.example.com'; $cdn2 = 'http://cdn2.example.com'; $cdn_extensions = array('css', 'js'); if($this_file_extension in $cdn_extensions){ $uri = $cdn1 . '/' . $path; } else{ $uri = $cdn2 . '/' . $path; }}
File StorageThird-party module - Storage API
● Save to FTP / HTTP● Save to Database● Save to S3● Save to Rackspace
Database Scaling
MongoDB? PostgreSQL?
Database ScalingDrupal 6 - happy querying, tragedy scaling
function statistics_get($nid) {
if ($nid > 0) { // Retrieve an array with both totalcount and daycount. $statistics = db_fetch_array(db_query('SELECT totalcount, daycount, timestamp FROM {node_counter} WHERE nid = %d', $nid)); }
return $statistics;}
Database ScalingDrupal 7 – DB abstract layer
$statistics = db_select('node_counter', 'n') ->fields('n', array( 'totalcount', 'daycount', 'timestamp')) ->condition('nid', $nid,'=') ->execute() ->fetchAssoc();
● Support another Database (not MySQL only)● Separate R/W query easily
Database Scalingrandom slave every time DB bootstrap
# default master (read / write query)$databases['default']['default'] = $info_array;# multiple slave (read only query)$databases['default']['slave'][] = $info_array;$databases['default']['slave'][] = $info_array;
Database Scalingpage specific query to slave by 1 click
Database Scaling
3. why Scalability matter?
我不胖，只是腫了一點
Not Fastest solutionBut Available solution
Why a CMS designed like this?● Pro
– Quick and easy to enter Drupal (even not Engineer)– Can stack special requirement into Drupal– When more function or more user, scale out
● Cons– Not so easy (if you would like to develop with D)– Not so flexible vs framework (because it isn’t)– Definitely can scale if well planned, but always not
沒有深深愛過怎知好與壞？我知道你胖，但還是愛你
you may interested in:● 2.4 million page views per day in Drupal
http://sf2010.drupal.org/conference/sessions/24-million-page-views-day-60-m-month-one-server.html
● Auto Scale Drupal setup in AWShttp://www.slideshare.net/burgerboydaddy/scaling-drupal-horizontally-and-in-cloud
● Drupal vs Djangohttp://birdhouse.org/blog/2009/11/11/drupal-or-django/
● Drupal with nodejshttps://www.drupal.org/project/nodejs
● Drupal with Dockerhttps://github.com/ricardoamaro/docker-drupal
● Drupal with MongoDBhttps://www.drupal.org/project/mongodb
Thank You!
每週三晚上 8:00 Hangout 網路聚
1. DrupalTaiwan.org
2. goo.gl/PxuhqQ
3. FB/groups/drupaltaiwan/DrupalTaiwan Facebook 社團
You can also find Drupaler here: