drupal camp london 2014 john ennew concurrency session deeson online

Download Drupal camp london 2014 John Ennew Concurrency Session Deeson Online

Post on 11-Jul-2015

829 views

Category:

Technology

0 download

Embed Size (px)

TRANSCRIPT

  • C O N C U R R E N C YA N I N T R O D U C T I O N T O

    J O H N E N N E W, S O L U T I O N S A R C H I T E C T, D E E S O N O N L I N E C E N G O N D . O . @ D E E S O N _ L A B S H T T P : / / W W W. D E E S O N - O N L I N E . C O . U K / L A B S

    O R H O W I M A D E M I G R AT E FA S T E R

  • W H AT I S C O N C U R R E N C Y ?

  • DatabaseFilestore

    Web service

    Application

  • DatabaseFilestore

    Web service

    Application

  • B R E A K I N G T H E P R O B L E M D O W N

  • E D W A R D A . L E E

    a folk definition of insanity is to do the same thing over and over again and to expect the results to be different. By this definition, we in fact require

    that programmers of multithreaded systems be insane. Were they sane, they could not understand

    their programs

    T H E P R O B L E M W I T H T H R E A D S , 2 0 0 6

    T H E H T T P : / / W W W. E E C S . B E R K E L E Y. E D U / P U B S / T E C H R P T S / 2 0 0 6 /E E C S - 2 0 0 6 - 1 . H T M L W I T H T H R E A D S , 2 0 0 6

  • T H E D I N I N G P H I L O S O P H E R S

    H T T P : / / D E E S O N - O N L I N E . C O . U K / L A B S / M U LT I - P R O C E S S I N G - PA R T- 1 -H O W - M A K E - D R U S H - R U S H

  • lock_acquire()

    drush_invoke_process()

    DB transactions (innodb)

    D R U PA L 7 T O O L S

  • M A K I N G M I G R AT E M O V E

  • W H Y S P E E D U P M I G R A T I O N S ?

    Reduce the time a migration takes

    Discover and fix problems quicker

    Remigrate faster / regular migrations

  • T H E P R O C E S S

    https://github.com/johnennewdeeson/drush-multi-processing

    Download mt.drush.inc and place in sites/all/drush

    Update to migrate 7.x-2.6-rc1

    Configure your migration class for batched operation

    Create a custom drush import command (or modify mtm.drush.inc also at github URL above)

    Full instructions online at: http://deeson-online.co.uk/labs/multi-processing-part-2-how-make-migrate-move

  • Configuring your migration class

  • Configuring your migration class

  • Creating a Drush command

  • Starting the process thread manager

  • Process setup

  • Process teardown

  • L I V E D E M O

  • F I N E T U N I N G

    Ramp up slowly - monitor server load and db load

    /etc/my.cnfinnodb_locks_unsafe_for_binlog = 1innodb_flush_log_at_trx_commit=2

    Use alternative cache mechanism to ease your database load (e.g. memcache)

    $conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';

    $conf['memcache_stampede_protection'] = TRUE;

  • T H A N K S ! A N Y Q U E S T I O N S ?

    J O H N E N N E W, S O L U T I O N S A R C H I T E C T, D E E S O N O N L I N E C E N G O N D . O . @ D E E S O N _ L A B S H T T P : / / W W W. D E E S O N - O N L I N E . C O . U K / L A B S