building big websites with drupal
DESCRIPTION
A short talk given to the Drupal Show and Tell meetup, 24 October 2013. Based on their experience as developers in Capgemini UK Digital, Andy Read covers a high level overview of architectures for high performance websites and Temi Jegede looks at some of the team dynamics issues.TRANSCRIPT
Building Big WebsitesDrupal Show and Tell, Thu Oct 24, 2013
Who Are We?• Andy Read ([email protected])
• Temi Jegede ([email protected])
(140,000 people)
(140 people)
Drupal Sites
Big Sites: It’s Not Rocket Science
It’s Much, Much Harder!
1 billion users
My Big Site 101
Dozens of Servers
Numerous 3rd-party services
3 sec round trip!!
Thrust
Arc
GravityJust follow the laws of physics Virtually infinite complexity!
Rockets 101
3 day round
trip
Goals• Scale (number of concurrent users)• Performance (fast response time)• Functional Scope• Integration with 3rd-party services• Team size and many roles (Dev, Test, Ops, BA...)
• Testability / Reliability / Reproducibility• Security & Corporate governance (reputation)
Drupal / LAMP ArchitectureDrupal
MySQL
Apache(Web Server)
Linux
PHP
File system
Single [virtual]
server
DB Server
A Bit More Performance
Drupal
MySQL
Apache /
Nginx
Linux
PHP
File system
Varnish cache
MemCacheSSD
RAM+++
APC
Serious Performance!Master DB
MySQL
Varnish cache
DrupalApache /
Nginx
Linux
PHP
File systemMemCache
SSDRAM+++
Slave DB
MySQL
Slave DB
MySQL
Slave DB
MySQL
DrupalApache /
Nginx
Linux
PHP
File systemMemCache
SSDRAM+++
DrupalApache /
Nginx
Linux
PHP
File systemMemCache
SSDRAM+++Drupal
Apache / Nginx
Linux
PHP
File systemMemCache
SSDRAM+++
Varnish cache
Load Balancer Servers ++
SAN Network storage
Yet More AnalyticsMore Analytics
We’re Not Alone!Drupal
High-Performance
Mega-Site
Product Catalogue
Postcode Address
Identity Check
Payment Gateway
SalesForce
Adobe PDF
ESB
Google Analytics
JSON / REST
XML/WSDL
Queue
Graylog
SMTP
SOAP
But...
Development Process
Dev VM SIT UAT ProdDev
VMDev VM
Pre-Prod
Git / SVN
Test, Test, Test, Unit Test, BDD (Behat), Manual Test...
Automate everything! Review, Build, Test & Deploy
User Data (!)Test Data
Code & ConfigurationContent(Editors &
Contributors)(Drupal 8 FTW!)
Team Dynamics
The way a team plays as a whole determines its success. You may have the greatest bunch of individual stars in the world, but if they don't
play together, the club won't be worth a dime.Babe Ruth
Small Site -> Small Team
• A Developer• Client• Tools (A dev environment, test/production server)
Multitasking
Big Site -> Big TeamMultiple different roles within a team:
Developers
Testers
Project managers
Business analyst
Dev-OPS
Designers
Web authors Clients
It can all get very crowded
GOLDEN RULE: Fragmented job roles but not a
fragmented team
In a perfect situation each team member should have a solid understanding of their job role and also a good grounding and
understanding of other team members’ job roles.
Tools
The Blind Men and The Elephant
Different people with different roles and perspectives will havedifferent ideas of what the “whole” is... Since each person can’t see the whole, we have to have good communication with each other to understand what the whole really is.
Links
http://gettingpredictable.com/what-elephants-teach-us-about-software-development
The information contained in this presentation is proprietary.© 2013 Capgemini. All rights reserved.
Rightshore® is a trademark belonging to Capgemini.
www.uk.capgemini.com
About CapgeminiWith more than 125,000 people in 44 countries, Capgemini is one of the world's foremost providers of consulting, technology and outsourcing services. The Group reported 2012 global revenues of EUR 10.3 billion.
Together with its clients, Capgemini creates and delivers business and technology solutions that fit their needs and drive the results they want. A deeply multicultural organization, Capgemini has developed its own way of working, the Collaborative Business ExperienceTM, and draws on Rightshore®, its worldwide delivery model.
Learn more about us at www.capgemini.com
... and We’re Hiring