large scale drupal - behind the scenes

Large Scale Drupal in

the Municipality of

Copenhagen - Behind

the Scenes

Drupal Camp Berlin 2014

About me

• Boyan Borisov

• Open Source Tech Lead

and Architect @

Propeople

• boyanb@propeople.dk

• @boyan_borisov

• Skype: boian.borisov

• https://drupal.org/u/bo

yan.borisov

What were the client’s requirements?

• Multisite platform able to handle a thousand web sites

• Scalable infrastructure able to handle a thousand sites available 24/7

• Multi vendor environment which allows every module to be reusable

• Content sharing between all the sites on all the profiles

• Centralized Digital Asset Management (DAM) system

What were the client’s requirements?

• Editorial friendly UI

• Hierarchy structure of the site pages

• Web Content Accessibility Guidelines

(WCAG) – WCAG 2.0 AA

• Performance

• Automated tests

• Continuous integration

Multisite platform able to handle a thousand web sites

One Aegir to rule

them all!

• What is Aegir?

• Why did we choose it?

• What did we do in

order to extend Aegir?

Scalable infrastructure able to handle a thousand sites available 24/7

Monitoring and logging

• Graphite - Scalable Realtime Graphing

• Elasticsearch - http://www.elasticsearch.org

• Logstash - http://logstash.net

• JSONlog - https://www.drupal.org/project/jsonlog

• Elasticsearch Connector -

https://www.drupal.org/project/elasticsearch_con

nector

Multi vendor environment which allows every module to be reusable

• Gitlab - https://about.gitlab.com

• Mini “drupal.org” for all municipality custom modules

• Every module into its own repository

• Drush make file to build a platform

• Reusable drush make files

• Phing build tool

• Jira

Reusable make files

core = 7.x

api = 2

; KKMS inheritance

includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/kkms_profile/raw/7.x-1.26/release.make

includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/kkdk_profile/raw/dev/contrib.make"

includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/kkdk_profile/raw/dev/features.make"

includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/kkdk_profile/raw/dev/custom.make"

Content sharing between all the sites on all the profiles

• Push content to a site(s)

• Push content to a channel

• Pull content from a site

• Pull content from a channel

• Centralized Digital Asset Management

(DAM) system

• Central content management system

Content sharing architecture

Centralized Digital Asset Management (DAM) system

Central Localization Distribution (CLD)

CLD

Site1 Site2 SiteN

Editorial friendly UI

• Field group • Media and Media browser plus • Admin views and Views bulk operations • Workbench• Entity reference and References dialog • Panelizer• Node clone • Ckeditor and Better formats • Chosen, Multiselect and Hierarchical Select • Taxonomy manager

Hierarchy structure of the site pages

• Be aware not to use a module which will

make a hierarchy on top of the menu_router

table!

• Solution is Menu Reference -

https://www.drupal.org/project/menu_refe

rence

The German trace

Web Content Accessibility Guidelines (WCAG) – WCAG 2.0 AA

● Drupal and accessibility

https://groups.drupal.org/node/18595

https://www.drupal.org/project/node_accessi

bility

● Test tools

http://siteimprove.com/

http://wave.webaim.org/

Accessibility Developer Tools - Chrome plugin

Performance

• Varnish with a custom vcl file

• Memcache

• Apache Solr views

• APC cache

• Horizontal and vertical scaling

Automated tests

Continuous integration

• Continuous integration with Aegir

• Gitlab + webhooks + merge request

• Jira + Gitlab integration

• Phing build tool for local development

• Release branches

• Jenkins

• Fabric

The best part!

What did we give back to the

community?

Contributions

● 90+ patches

● 300+ comments

● 7 contributed modules

Questions?