php projects beyond the lamp stack
TRANSCRIPT
projectsBeyond the LAMP stack
By Thijs Feryn
PHP
Hi, I’m Thijs
I’m @ThijsFeryn on Twitter
I’m an Evangelist
At
I’m a at
board member
LAMPStack?
CPU memory I/O
PHP consumes lots of
Faster PHP
Different approach
Offloading
•Webserver •Database •User process
Caching
Optimize database
Optimize runtime
Avoid
Avoid
Don’t recompute
data that hasn’t
changed
Offload the webserver
Varnish
Reverse proxy
User Varnish Server
Proxyinthedatacenter
✓ Varnish Configuration Language ✓ Edge Side Include support ✓ Gzip compression/decompression ✓ Cache purging ✓ HTTP streaming ✓ Grace mode ✓ Configure backends ✓ Backend loadbalancing ✓ ACL protection ✓ VMODs in C
Varnish
Extend the default behavior
in VCL
✓Strip tracking cookies (Google Analytics, …) ✓Sanitize URL ✓URL whitelist/blacklist ✓PURGE ACLs ✓Edge Side Include rules ✓Alway cache static files ✓Extend hash keys ✓Override TTL ✓Define grace mode
What to extend?
vcl 4.0; import std;
acl purge { "127.0.0.1" }
backend default { .host = "176.62.160.59"; .port = "80"; .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; }
sub vcl_recv {
if (req.http.Authorization) { return (pass); }
if (req.http.Accept-Encoding) { if (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; } else { unset req.http.Accept-Encoding; } } if (req.method == "PURGE") { if (!client.ip ~ purge) { return(synth(405,"Not allowed."));
Minimal VCL
Let the application
handle it
Caching in your architecture
Respect HTTP
Cache-control: public, max-age=3600, s-maxage=7200
Cache-control: no-cache, no-store
VS
Cookies?
State
Non-cacheable
content
Edge Side Includes
<!DOCTYPE html><html><head> <title>The Demo</title> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="/css/bootstrap.min.css"> <link rel="stylesheet" href="/css/bootstrap-theme.min.css"> <script src=“/js/jquery-2.1.4.min.js"></script> <script src="/js/bootstrap.min.js"></script></head><body><nav class="navbar navbar-inverse navbar-fixed-top"> <esi:include src=“http://mysite.dev/nav" /></nav>
<div class="jumbotron"> <esi:include src="http://mysite.dev/jumbotron" /></div>
<div class="container"> {% block content %}{% endblock %} <hr> <footer> <esi:include src="http://mysite.dev/footer" /> </footer></div></body></html>
ESI tags
Or just use AJAX
Async Graceful degradition
✓Cache pages and static assets ✓Fastest way ✓Hit rate may vary ✓Chop your content up in pieces ✓Use ESI or AJAX ✓Gateway to your application
Where does Varnish fit in?
You can also host your
static files on a separate set
of Nginx servers
Content Delivery Network
CDNs are nothing but a
bunch of reverse caching proxies
Put the content where your
user is
Offload the database
Data is stored for flexibility, not for
performance
SQL (joins) allow different
compositions of the same data
Make the data retrieval faster
✓ Key-value store ✓ Fast ✓ Lightweight ✓ Data stored in RAM ✓ ~Memcached ✓ Data types ✓ Data persistance ✓ Replication ✓ Clustering
Redis
Redis$ redis-cli 127.0.0.1:6379> ping PONG 127.0.0.1:6379> set mykey somevalue OK 127.0.0.1:6379> get mykey "somevalue
✓ Strings ✓ Hashes ✓ Lists ✓ Sets ✓ Sorted sets ✓ Geo ✓ …
Redis data types
Redis$ redis-cli 127.0.0.1:6379> hset customer_1234 id 1234 (integer) 1 127.0.0.1:6379> hset customer_1234 items_in_cart 2 (integer) 1 127.0.0.1:6379> hmset customer_1234 firstname Thijs lastname Feryn OK 127.0.0.1:6379> hgetall customer_1234 1) "id" 2) "1234" 3) "items_in_cart" 4) "2" 5) "firstname" 6) "Thijs" 7) "lastname" 8) "Feryn" 127.0.0.1:6379>
$ redis-cli 127.0.0.1:6379> lpush products_for_customer_1234 5 (integer) 1 127.0.0.1:6379> lpush products_for_customer_1234 345 (integer) 2 127.0.0.1:6379> lpush products_for_customer_1234 78 12 345 (integer) 5 127.0.0.1:6379> llen products_for_customer_1234 (integer) 5 127.0.0.1:6379> lindex products_for_customer_1234 1 "12" 127.0.0.1:6379> lindex products_for_customer_1234 2 "78" 127.0.0.1:6379> rpop products_for_customer_1234 "5" 127.0.0.1:6379> rpop products_for_customer_1234 "345" 127.0.0.1:6379> rpop products_for_customer_1234 "78" 127.0.0.1:6379> rpop products_for_customer_1234 "12" 127.0.0.1:6379> rpop products_for_customer_1234 "345" 127.0.0.1:6379> rpop products_for_customer_1234 (nil) 127.0.0.1:6379>
Redis
Clustering
✓ Database/API cache ✓ PHP session storage ✓ Message queue (lists) ✓ NoSQL database ✓ Real-time data retrieval
Where does Redis fit in?
Basically: Real-time
& volatile data
BUT:
MySQL can still remain “source of truth” database
✓Full-text search engine ✓Analytics engine ✓NoSQL database ✓Lucene based ✓Built-in clustering, replication, sharding ✓RESTful interface ✓Schemaless
ElasticSearch
POST /my-index {"acknowledged":true}
POST/my-index/my-type { "key" : "value", "date" : "2015-05-10", "counter" : 1, "tags" : ["tag1","tag2","tag3"] }
{ "_index": "my-index", "_type": "my-type", "_id": "AU089olr9oI99a_rK9fi", "_version": 1, "created": true }
Confirmation
GET/my-index/my-type/AU089olr9oI99a_rK9fi?pretty
{ "_index": "my-index", "_type": "my-type", "_id": "AU089olr9oI99a_rK9fi", "_version": 1, "found": true, "_source": { "key": "value", "date": "2015-05-10", "counter": 1, "tags": [ "tag1", "tag2", "tag3" ] } }
Retrieve document by
id
Document & meta data
Analyzed vs
non-analyzed
Full-text vs
exact value
Filter vs
Query
Search
POST /products/product/_search?pretty { "query": { "match": { "name.raw": "Linen Blazer" } } }
POST /products/product/_search?pretty { "query": { "filtered": { "query": { "match_all": {} }, "filter": { "term": { "name": "Linen Blazer" } } } } }
Matches 2 products
Matches 1 product
Aggregations
Group by on steroids
POST /products/product/_search?pretty { "fields": ["category","price","name"], "query": { "match": { "name.raw": "blazer" } }, "aggs": { "avg_price": { "avg": { "field": "price" } }, "min_price" : { "min": { "field": "price" } }, "max_price" : { "max": { "field": "price" } }, "number_of_products_per_category" : { "terms": { "field": "category", "size": 10 } } } }
Multi-group by &
query
"aggregations": { "min_price": { "value": 455 }, "number_of_products_per_category": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "Blazers", "doc_count": 2 }, { "key": "Default Category", "doc_count": 2 }, { "key": "Men", "doc_count": 2 } ] }, "max_price": { "value": 490 }, "avg_price": { "value": 472.5 } }
Aggregation output
Clustering
✓ Full-text search engine with drill-down search
✓ Human language & text analysis
✓ NoSQL database ✓ Big data analytics tool
using Kibana
Where does ElasticSearch fit in?
Offload the user process
Worker scripts
✓ Uses PHP-CLI ✓ Runs continuously ✓ Process forking ✓ Pthreads ✓ Run worker scripts in
parallel ✓ Managed by supervisord
Worker scripts
✓ Sync MySQL & Redis/ElasticSearch
✓ Resize images ✓ Async logging & metrics ✓ Other tasks that shouldn’t
block the user process
Worker scripts
Message queues
✓ Pub/sub ✓ Speaks AMQP protocol ✓ Supported by Pivotal ✓ Channels/Exchanges/
Queues ✓ Built-in clustering ✓ Reliable messaging
RabbitMQ
✓ Take load away from user process
✓ Free up resources on frontend servers
✓ Elaborate messaging strategies
✓ Async event-based actions
Where do RabbitMQ/workers fit in?
But with most of these tools we still
go through the PHP runtime …
SOA Microservices
AJAX Websockets
✓ Javascript runtime ✓ Async ✓ Event-driven ✓ Non-blocking I/O ✓ Callbacks ✓ Lightweight ✓ NPM packages ✓ Backend-code in Javascript
NodeJS
const http = require('http');
const hostname = '127.0.0.1'; const port = 1337;
http.createServer((req, res) => { res.writeHead(200, { 'Content-Type': 'text/plain' }); res.end('Hello World\n'); }).listen(port, hostname, () => { console.log(`Server running at http://${hostname}:${port}/`); });
Gateway to ElasticSearch,
RabbitMQ & Redis
✓ Compiled language for the web ✓ Invented by Google ✓ Strictly typed ✓ Feels like your average
interpreted language ✓ Async features ✓ Built for systems programming ✓ REALLY fast ✓ Not object oriented
Go(lang)
package main
import ( "fmt" "net/http" "goji.io" "goji.io/pat" "golang.org/x/net/context" )
func hello(ctx context.Context, w http.ResponseWriter, r *http.Request) { name := pat.Param(ctx, "name") fmt.Fprintf(w, "Hello, %s!", name) }
func main() { mux := goji.NewMux() mux.HandleFuncC(pat.Get("/hello/:name"), hello) http.ListenAndServe("localhost:8000", mux) }
Really fast workers Really fast APIs
Could replace PHP
workers
Could replace NodeJS
The end game
✓ Cache pages (Varnish) ✓ Assemble content via ESI or AJAX ✓ Static assets on Nginx or CDN ✓ Business logic in lightweight API calls
(NodeJS, Go) ✓ Key-value stores for volatile & real-time
data in Redis ✓ ElasticSearch as a NoSQL database ✓ RabbitMQ for async communication ✓ Worker processes read from message
queue
End game
Use the right tool for the job
Use the tools you like
