introduction to magento optimization

Introductionto Magento Optimization

Because sometimes “Copy&Paste” isn’t enough

Fabio Daniele

@fabx2dan@synesthesiait

Why are we here?

This is a set of slides which will introduce you to the magical, overflowed, strenuous world of Magento optimization.

It won't be problem-centric, but a quick tour on the whole ensemble of Mage's good practices to achieve excellent performance and understand why this platform has such a moody character.

Ok then: what makes it so slow?

Magento, in fact, isn’t that slow.

The problem basically resides in its fragile scalability, due to very low tolerance for bad code, the EAV model which generates performance-killer queries, an excessive dependence on media resources and a dispersive class structure.

Ok then: what makes it so slow?

Descending more into details, we can split the area of operation into five sections:

➢ HTTP Transactions➢ Database➢ Media➢ Source code➢ Tweaking

Each one of them has its own peculiarity, with specific "dos" and "don'ts". Let’s take a look.

HTTP Transactions

The main responsible for client-server communication, its performance greatly impacts the user experience on the website.

The best optimization available can be made using a caching system. There are different kind of them, the choice will be influenced by the load of the website and the scope of the cache. We will discuss this further.

But to rely only on cache for optimization is a bad mistake. Caching is useful to push optimization over the boundaries only when there's no other option left.

Database The major drawback of the EAV model it's the extremely dispersive structure, which generates queries with lots of JOINs.

Besides, the excessive amount of transparency of the Model to the eyes of the developer, and its counterintuitive nature, may lead to bad code application, affecting performance while interacting with the database.

Database Other than best practices, we can achieve major advantages through replication since the structure generated by the EAV suffers from really bad concurrency illness.

Replication is most effective on tables with an high access rate, typically the FLAT ones. This form of optimization can be implemented manually or through 3rd party services, like the ones offered by AWS.

Media With the term media we are mainly referring to images. Those are often managed by the customer itself, leading to possible inefficiencies in size and quality of the images.

Without the proper compression, a single image may take up to 80% more bandwidth than an optimized one! Even when Magento responds quickly, if the browser needs to download lots of MBs of images the user will still get a sensation of slowness.

Media There are services and plugins for dealing with the optimization without having to check manually every image uploaded to the platform.

The most famous are Kraken.io and Apptrian Image Optimizer, which intercept the images being requested, call a webservice for optimizing size (with both lossy or lossless algorithms) and then return it, properly cached.

Assets Just like the media, assets have great impact on the loading time of the client. The best option here is to relocate and minify the code.

To relocate all the js and css source in the same host is useful for reducing network time over the resolution of DNS and routing to the server, while the minification reduces the size of the single files and the number of requests needed to download them all.

One of the most used extension is Apptrian Minify HTML CSS JS.

Source code The possibilities to extend Magento functionalities are nearly limitless, but this flexibility costs much.

Given the chance to use Models for interacting with database, it's essential to know exactly how code operates, in order to prevent improper use of objects and classes which may lead to overheads of JOINs or unwanted big queries for just small chunks of information.

Source codeIteration $p = Mage::getModel('catalog/product')

->getCollection();

foreach ($p as $item) { // Doin’ stuff...}

This is a don’t, as Magento uses a lazy loading approach while dealing with collections, thus loading the entire PHP object at every cycle of the for. It is utterly expensive.

Source codeIteration $p = Mage::getModel('catalog/product')

->getCollection();

foreach ($p->getData() as $data) { // Doin’ stuff}

This is the proper way, because it loads only the informations we really need.

Source codeIteration $p = Mage::getModel('catalog/product')

->getCollection();

while ($item = $p->fetchItem()) {// Doin’ stuff

}

This is totally analogue to the previous solution, with just some saving in the amount of memory stored for each item of the collection.

Source codeIteration

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24

Time Memory

wrong way ~ 0.0366 sec ~ 3.14 MB

getData() ~ 0.0052 sec ~ 1.16 MB

fetchItem() ~ 0.0211 sec ~ 0.30 MB

Source codeScaling $p = Mage::getModel('catalog/product')

->getCollection();

foreach ($p as $item) {$item->load($item->getId());

echo $item->getDescription();}

This is a don’t, as at every cycle we are loading the entire product object just for a single information.

Source codeScaling

$p = Mage::getModel('catalog/product')->getCollection()->addAttributeToSelect(

array('description'));

foreach ($p as $item) { echo $item->getDescription();}

Here we are correctly specifying which information we are going to need later, lightening the execution of the cycle.

Source codeScaling

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24

Time Memory

without attribute ~ 17.07 sec ~ 25.83 MB

with attribute ~ 0.0435 sec ~ 3.45 MB

Source codeSaving $p = Mage::getModel('catalog/product')

->load($id);

$p->setDescription('New Test')->save();

Another don’t. This way of saving forces Magento to store the entire object, not just the attribute we’ve modified.

Source codeSaving $p = Mage::getModel('catalog/product')

->load($id);

$p->setDescription('New Test')->getResource()->saveAttribute($p, 'description');

Instead of storing the entire object, we specify exactly the attribute to be saved, thus sparing us time and memory.

Source codeSaving

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24

Time Memory

save() ~ 0.336 sec ~ 5.99 MB

saveAttribute() ~ 0.0432 sec ~ 2.47 MB

Source codeReuse & Recycle

# Multiple static call$nam = Mage::getModel('catalog/product' )->load($id)->getName();$sku = Mage::getModel('catalog/product' )->load($id)->getSku();$des = Mage::getModel('catalog/product' )->load($id)->getDescription ();

/******** VERSUS ********/

# Single static call$p = Mage::getModel('catalog/product' )->load($id);$nam = $p->getName();$sku = $p->getSku();$des = $p->getDescription ();

It may seems obvious, but attention to details is needed when working with big amount of data.

Source codeReuse & Recycle

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24

Time Memory

multiple ~ 0.0823 sec ~ 2.54 MB

single ~ 0.0376 sec ~ 2.54 MB

Thinking of a store with about 10.000 products, performance would be impacted greatly.

Source codeNative funcs

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24

Time Memory

count() ~ 0.0363 sec ~ 3.14 MB

getSize() ~ 0.0019 sec ~ 0.0016 MB

Sometimes we found ourselves thinking that native PHP functions should work better than the ones offered by a framework. This is true in some cases, but not with Magento. Here’s an example: counting elements in a collection.

Cache The temptation of using a cache system to speed up our website is very strong, but we shall not fall for the Dark Side.

Instead, caching is effective only after all the other optimizations have been applied.

Moreover, regardless of the cache system that one will choose for its website, it's very important to plan accurately and think over the pros and the cons of using the proper system, if you are going to need it at all.

CacheIn Mage World, we can distinguish two major cache family:

➢ Infrastructure, server-level

➢ Application, managed by Magento itself

CacheInfrastructure

This type of cache uses services implemented outside the basecode, typically reverse proxies, which take place between the client and the Magento application entry point.

It greatly reduces the response time, because when a request hits the cache Magento hasn’t been instanced yet and all the callback effort is handled by the proxy service itself.

Besides, reducing the number of hit to Magento will also reduce the amount of load on the server.

CacheInfrastructure

The most famous reverse proxy is Varnish, widely used for eCommerce sites with high traffic levels.

It also provides support for a specific hypertext syntax called ESI: it’s used to specify, through proper tags, which content of the page needs to be reloaded every time, committing the job to Varnish.

When the proxy encounters an ESI tag, it will send a request to the application to obtain the resource, merge the result with the cached response and then return it to the client.

CacheApplication

The second cache family has a more diversificated situation. Many different extensions have been implemented from time to time, all of which different from the others by the way the cache is implemented (before or after Mage instance) or the the kind of storage being made (full, partial).

So, for the sake of orderliness, we will split up this types in three:

❖ Full Page Cache❖ Static Cache❖ Hybrid Cache

CacheFull Page Cache

Full Page Cache (FPC for friends) is intended for caching the most out of a page, while reloading the resource just for specific chunks of page that are requested as fresh every time.

The advantage of this extension is that it kicks in before all the Mage app object is instanced, thus saving lots of resources.

The drawback, however, resides in the necessity to parse the response, which can be enormous, and to retrieve the dynamic content to replace in the cached response for every request.

CacheStatic Cache

This kind of cache is the most effective, but its range of use is very limited. Like FPC, it is placed before the instance of Mage app and it caches all the content of the request, regardless of the content.

The cons, here, are clear: it can't be used in any page that provides dynamic objects, like a cart, or session-related informations.

However it is very useful with static pages like 404 (which are very expensive in Magento!), external widgets or other static contents (like a 'Contact us' block).

CacheHybrid Cache

The Hybrid tries to take the pros of both types, reducing the cons.

The content is returned through Static Cache, so it's faster and the user will be served asap, while to the response we will attach javascript algorithm to create asynchronous calls towards the server and retrieve the dynamic content after the page has been loaded.

This approach will return only the information needed, while the async calls may be cached again as they are fully compatible with any other cache system, both Application and Infrastructure.

Tweaking Usually snobbed by developers for small or mid-sized application, a proper LAMP/LNMP configuration setup becomes mandatory working with Magento.

In this presentation we will list only the most important tweaks, but for a complete list I suggest to follow this link.

TweakingServer

1. Turn on the Gzip compression: it reduces output size, speeding up the download for clients.

2. Turn on the option "KeepAlives" of Apache: this option enables the server to channel multiple HTTP request through a single TCP connection, improving response time. Be careful, though, because setting a wrong time for this parameter may lead to an excess of open TCP connections, which will increase considerably the load on the server.

TweakingServer

3. Use a RAM filesystem for storing files with an high rate of I/O operation, usually var/cache and var/session.

4. Reduce logging operation when not needed: it's an I/O that may slow down HTTP requests; alternatively, think of an async write or the use of a memory based filesystem.

TweakingPHP

1. Use an accelerator like APC, Zend Opcache or XCache.

2. Increase the memory_limit to at least 128M

3. Check the real amount of realpath cache used by PHP and set the correct amount in the realpath_cache_size option. You can verify it using the realpath_cache_size() function.

TweakingPHP

4. Turn off open_basedir, as it prevents realpath cache use.

5. Use an Autoloader, which will reduce I/O operation while scavenging for classes during Mage execution.Another ally in the parse of the base code may be the AOE_ClassPathCache extension, which does not create an Autoload file but registers all the filepaths for resolving classes requests into a single file, thus saving time for scanning all the dirs searching for the proper file.

TweakingDatabase

1. Disable log writing to database if not needed. Magento does not offer this option by default, it can be achieved through the use of an extension called Yireo DisableLog.

2. Increase the query_cache_size amount, useful when server comes with loads of RAM. MySQL packages tend to be resource conservative by default. It can be increased to 64MB if the server has at least 1GB of RAM.

3. Increase the query_cache_limit, bringing it to at least 2MB.

TweakingDatabase

4. MySQL uses a buffer to store information on data and indexes, to speed up queries execution.The buffer size can be monitored with the innodb_buffer_pool_size option, which value depends on the database server specs.

Database server RAM SizeShared with HTTP server 6 GB 2/3 GB

Dedicated 6 GB 5 GBDedicated 12 GB 10 GB

TweakingDatabase

5. Increase the innodb_thread_concurrency attribute, which defines the maximum number of concurrency threads inside InnoDB. By default this option is set to 0. It's useful for websites with at least 60 active user at the same time.A good way to guess the correct amount of concurrency threads is to apply this formula:

(2 * [n° of CPU])+ 2

Platform This is not proper optimization, but it’s a good point to consider when talking about performance.

The platform over which Magento runs can greatly improve speed, stability and scalability.

One of the best booster for a Magento 1.9 installation would be a setup composed by PHP 7 FPM and MySQL Percona 5.6. Advantages are tremendous, and will be shown in comparison with past benchmarks.

Platform Let’s take the code of page #16, which is the heaviest of all in this presentation.

Time Memory

PHP 5.6* ~ 17.07 sec ~ 25.83 MBPHP 7** ~ 6.28 sec ~ 13.12 MB

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

*Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24vs.

**Vagrant - OS Ubuntu 14.04 - PHP 7.0.0 FPM - MySQL Percona 5.6.27

Platform We can achieve interesting results also with code from #19, during the save operation of a product. In this case, Mage cache was not taken in account.

Time Memory

PHP 5.6* ~ 0.643 sec ~ 7.576 MBPHP 7** ~ 0.310 sec ~ 3.465 MB

Benchmark made on a Magento 1.9 installation with official sample data (~580 products)

*Vagrant - OS Ubuntu 14.04 - PHP 5.6.12 with APC - MySQL 5.6.24vs.

**Vagrant - OS Ubuntu 14.04 - PHP 7.0.0 FPM - MySQL Percona 5.6.27

Platform While officially Magento 1.9 does not support PHP 7, there are simple fixes that needs to be made in the Core in order to get it up and running.

The real problem is, no one can assure backward compatibility for Magento 1.x modules, thus potentially transforming a simple platform migration into your worst nightmare.

This kind of change should not be done on running and stable projects, but it’s a really good (and brave) start for new ones. Even starting from fresh, though, we need to be careful about code written for PHP 5 in 3rd party extensions.

At last...

Released under Creative Commons by-nc-nd

...thanks for watching!Synesthesia srl

HQ: Via Amedeo Peyron 29, 10143 Torino

Tel: +39 011 0437401 - info@synesthesia.it - www.synesthesia.it - www.droidcon.it

Reg. Office: C.so Galileo Ferraris 110, 10129 Torino - VAT N° IT10502360018

Mobile Apps // Web Development // Ecommerce & Magento // UX&UI Design