effective doctrine2: performance tips for symfony2 developers

Effective Doctrine2Performance Tips for Symfony2 Developers

Wrocław Symfony Group #2 April 24th, 2014

Effective Doctrine2: Performance Tips for Symfony2 Developers marcin.chwedziak

Who am I?


Symfony2 ContributorSilex Contributor

Symfony2 Developer

Angular.js D

eveloper

Ag

ile E

nthu

sias

t

„No worries, I

will handle th

is”

App

reci

ated

Con

trac

tor

Marcin Chwedziak github.com/tiraeth

lnkd.in/dp6sPzc

Software Engineer at Spork Labs LLP (Salt Lake City, UT)


What is this talk about?


plus things that can go wrong while working with Symfony2

107 queries executed in 17.25 seconds


While working with databases, the most important strategy is to avoid querying a database.


Rule #1: Caching


If you must query a database, be wise and do it smarter.


Rule #2: Code Optimization


Smart behaviour also means to be explicit in actions.


Rule #3: Transaction Demarcation


Caching w/ Doctrine2


Metadata

What can be effectively cached?

Query Result


Parsing model metadata from different sources with each request can have a serious impact on your application’s performance. It is a good practice to cache it.


Each time you write a DQL query or use the QueryBuilder to create one, Doctrine has to transform that query to SQL. This is an unnecessary waste of available resources.


Usually, there is no special reason to hit a database for rarely modified data. Configure Result Cache strategy to make sure that your database is not swamped with such queries.


Be aware!

Make sure you properly create your queries.


# Fetch expired subscriptions (non-cachable) SELECT s FROM ASBillingBundle:Subscription s WHERE s.validTo < ? [1397855248] !# Fetch expired subscriptions (cachable) SELECT s FROM ASBillingBundle:Subscription s WHERE s.validTo < CURRENT_TIMESTAMP()

You can cache both DQL and Native SQL queries.


How to cache metadata in Symfony2?


app/config/config_prod.yml doctrine: orm: # ... metadata_cache_driver: apc # ...


Can I implement custom caching service?


app/config/config_prod.yml doctrine: orm: # ... metadata_cache_driver: type: service id: as.doctrine.cache.couchbase # ...


src/AS/CoreBundle/Doctrine/Cache/CouchbaseCache.php <?php !namespace AS/CoreBundle/Doctrine/Cache; !use Doctrine\Common\Cache\CacheProvider; !class CouchbaseCache extends CacheProvider{ // doFetch($id) // doContains($id) // doSave($id, $data, $liteTime = 0) // doDelete($id) // doFlush() // doGetStats() }


How to cache queries in Symfony2?


app/config/config_prod.yml doctrine: orm: # ... query_cache_driver: apc # ...


How to cache query results in Symfony2?


app/config/config_prod.yml doctrine: orm: # ... result_cache_driver: apc # ...


Setting caching driver for Query Result Cache is not sufficient for enabling the strategy. You need to explicitly enable it for each and every query you’re interested in.


<?php !// ... !$query = $em->createQuery($dql); $query->useResultCache(true); !// …


With great power comes great responsibility.

Voltaire (1694-1778)


Be aware that high traffic websites can generate races while accessing cached data. This situation is called cache slam. When implementing own cache driver, please refer to your backend’s documentation regarding races avoidance.


If anything goes wrong, you can manually clear the cache without restarting/flushing the storage.

php app/console doctrine:cache:clear-metadata php app/console doctrine:cache:clear-query php app/console doctrine:cache:clear-result


Optimization Guidelines


You can begin your journey with code optimization while using Doctrine2 with properly set up domain models.

Please be aware that the following guidelines are for use with Doctrine2, not necessarily with database design in general.


Step 1: Avoid bidirectional associations


If you really need bidirectional associations, try to use lazy fetching strategy, but in most cases bidirectional relations are not necessary and should be avoided.


Be aware that lazy fetching strategy creates dynamic proxy objects (class is generated on the fly).

Fortunately, Symfony2 handles proxy generation with cache warmup task.


For aggregations (with Collections) you can use EXTRA LAZY strategy which will not load the whole collection on first access, as it is done in LAZY strategy.


Dropping such relations does not mean that you can’t use JOIN with your DQL / Native SQL queries.


<?php !namespace AS\BillingBundle\Entity; !class Subscription { /** * @ORM\ManyToOne(targetEntity="User") * @ORM\JoinColumn(name="user_id", referencedColumnName="id") */ protected $user; } !!!class User { // ... }


SELECT u FROM ASBillingBundle:User u JOIN ASBillingBundle:Subscription s WITH s.user = u WHERE s.validTo <= CURRENT_DATE()


Step 2: Avoid unnecessary cascades


1. Do not configure cascade operations on every association you have in your domain.

2. Unless required (e.g. additional file removal), use DBMS cascades instead of ORM ones.


Remember that you do not setup ORM cascades on target entites but on the triggering ones. Opposite to DBMS configuration.


<?php !namespace AS\BillingBundle\Entity; !class Subscription { /** * @ORM\OneToOne(targetEntity="User", cascade={"remove"}) * @ORM\JoinColumn(name="user_id", referencedColumnName="id") */ protected $user; }

This is wrong:


<?php !namespace AS\BillingBundle\Entity; !class Subscription { /** * @ORM\ManyToOne(targetEntity="User") * @ORM\JoinColumn( * name="user_id", * referencedColumnName="id", * onDelete="CASCADE" * ) */ protected $user; }

This is correct:


Step 3: Avoid lifecycle events abuse


Be smart about listening to lifecycle events like persist, update, or remove. Know the difference between Listener and Subscriber.

Do not identify Doctrine2 event subscribers with Symfony2 subscribers, which are more dynamic!


Avoid requiring EntityManager in your event listeners (it will just not work).


<?php !namespace AS\BillingBundle\Listener; !use Symfony\Bridge\Doctrine\RegistryInterface; !abstract class LifecycleListener { protected $doctrine; protected $manager; ! public function __construct( RegistryInterface $doctrine, $manager = 'default' ) { $this->doctrine = $doctrine; $this->manager = $manager; } ! protected function getEntityManager() { return $this->doctrine->getEntityManager($this->manager); } }


Step 4: Load from database what you need


Fetch Modes

Return only data you are in need of.

Hydration Modes

Partial Objects


When dealing with one-to-one and many-to-one relations, you can switch between EAGER and LAZY fetching modes right in your Query object.

<?php !$query = $em->createQuery( "SELECT s FROM ASBillingBundle:Subscription s" ) ->setFetchMode( "ASBillingBundle:Subscription", "user", \Doctrine\ORM\Mapping\ClassMetadata::FETCH_EAGER );


If not necessary, do not fetch a whole object (with relations) to just get its „validTo” field value.

<?php !$expirationDate = $em->createQuery( "SELECT s.validTo " . "FROM ASBillingBundle:Subscription s " . "WHERE s.user = ?1" ) ->setParameter(1, $user) ->getSingleScalarResult();


Unless you can deal with scalar values or arrays, you may need to wrap data in your model’s class. Use Partial Objects for this purpose.

<?php !$reference = $em->createQuery( "SELECT PARTIAL s.{plan,validTo} " . "FROM ASBillingBundle:Subscription s " . "WHERE s.id = ?1" )->setParameter(1, $id)->getResult(); !$reference->getPartialReference("ASBillingBundle:Subscription", $id);


Step 4: Load from database when you need


If you have to process data for large result sets, think about using DQL UPDATE/DELETE queries or Iterable Result.

When using Iterable Result always remember to detach all existing and not required anymore objects from Doctrine!


<?php !$batch = 50; $i = 0; !$iterableResult = $em->createQuery( "SELECT s FROM ASBillingBundle:Subscription s" )->iterate(); !foreach ($iterableResult as $row) { $subscription = $row[0]; $subscription->renew(); if (($i % $batch) === 0) { $em->flush(); // Execute queued updates $em->clear(); // Detach objects and GC them } ++$i; } !$em->flush(); // Execute remainig updates $em->clear(); // Detach remaining objects and GC them


<?php !$iterableResult = $em->createQuery( "SELECT s FROM ASBillingBundle:Subscription s " . "WHERE s.validTo = CURRENT_DATE() AND s.trialing = TRUE" )->iterate(); !foreach ($iterableResult as $row) { $mailer->notifyTrialExpiration($row[0]); $em->detach($row[0]); // Detach and GC immediately }


Transaction Demarcation


Proper transaction boundary definition can have positive impact on your application’s performance.

Doctrine2 takes care of proper implicit demarcation for UnitOfWork–handled objects and queue all operations until EntityManager#flush() is invoked.


However, Doctrine2 encourages developers to take over and control transaction demarcation tasks themselves.

Please remember to flush the EntityManager prior to transaction commit. In case of any exceptions, always close the EntityManager.


<?php !$em->getConnection()->beginTransaction(); !try { $user = new User; $subscription = new Subscription; $subscription->setUser($user); ! // ... do some work ! $em->persist($user); $em->persist($subscription); $em->flush(); $em->getConnection()->commit(); } catch (Exception $e) { $em->getConnection()->rollback(); $em->close(); ! // ... do something about the exception }


<?php !$em->transactional(function($em) { $user = new User; $subscription = new Subscription; $subscription->setUser($user); ! // ... do some work ! $em->persist($user); $em->persist($subscription); });

Using EntityManager#transactional() will automatically flush the manager prior to commit.


<?php !$em->getConnection()->transactional(function($em) { $user = new User; $subscription = new Subscription; $subscription->setUser($user); ! // ... do some work ! $em->persist($user); $em->persist($subscription); $em->flush(); });

Warning! This will not close EntityManager on exception.

Thank you!

Wrocław Symfony Group #2 April 24th, 2014

effective doctrine2: performance tips for symfony2 developers

Technology