NOW PLAYING LIVEDAEMONS IN PHP

We. Are.

1

.

1

DAEMONS IN PHP

Zoltán Németh

Core Systems Manager @ Ustream

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 2

CONTENTS

WHY???? The core: daemonizing Signal handling Init scripts Easy building of daemons The current package Open sourcing

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 3

WHY????

The need for daemons– Frequently running scheduled tasks– Async processing– Cron drawbacks:

Can do only minute level Does not guarantee exact timing Cron jobs cannot be monitored

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 4

WHY????

PHP had some problems in the early days– Hard to manage long running scripts– Memory leaks– Problems with forking– Handling code changes

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 5

WHY????

Why daemons in PHP?– Mostly PHP codebase– Duplication of business logic leads to hell

Lot of the logic codes used in daemons were already used on the site

– PHP experience– Quick and easy development

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 6

DAEMONIZING – THE CONCEPT

At startup, fork a child and exit from the parent– Start a new background session for the child

In the child, have a loop which does the task Calculate timing after each loop and use

usleep for precision Have a pidfile (for monit etc)

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 7

DAEMONIZING – THE CODE

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 8

DAEMONIZING – THE CODE

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 9

DAEMONIZING – THE CODE

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 10

THE MAIN LOOP

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 11

TIMING

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 12

MEMORY LEAK CHECKING

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 13

SIGNAL HANDLING

Before PHP 5.3– declare(ticks=1);– Signal dispatch by php system, at every tick– Unreliable– Precise, fixed intervals

Handler functions with pcntl_signal

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 14

SIGNAL HANDLING

PHP 5.3– declare(ticks=1); is deprecated– Signal dispatch by code:

pcntl_signal_dispatch– More reliable– Not precise, changing intervals (code

execution determines timing)Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 15

SIGNAL HANDLING

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 16

SIGNAL HANDLING

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 17

INIT SCRIPTS

Needed for monit, run daemons on startup, etc In the beginning: separate init script for every

daemon Lot of copy-paste with minimal changing

information Hard to keep consistent: changes do not

propagateMar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 18

INIT SCRIPTS

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 19

INIT SCRIPTS

Conclusion: let’s have a common script in the repo

Take the name of the daemon as a parameter

Required: a class to run a daemon by its name– Daemon_Runner

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 20

DAEMON RUNNER AND BUILDER

Original setup: one php file for each daemon Each has its own class definition

– All variable initializations– All configurations

Constructing the object and all dependencies Starting the daemon

Lot of boilerplate code Daemon_RunnerMar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 21

THE BOILERPLATE

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 22

DAEMON RUNNER AND BUILDER

Concept: a generic Runner class to run a daemon based with a given task

Task: the repeatedly running part of the daemon

Ustream_Daemon_Task interface Independent of the daemon configuration

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 23

DAEMON RUNNER AND BUILDER

Builder: a factory which creates the daemon object

All configuration options are handed over by the builder

Runner: call the builder and then run the resulting class

Configuration in .ini files Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 24

INI FILE EXAMPLE

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 25

BUILDER CALL, LIST OF OPTIONS

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 26

BUILDER CODE

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 27

AVAILABLE OPTIONS

Sleep: period length (running time is substracted)

Minimum sleep: if running time is too long Factory: Task factory class Instance: for multi instance daemons

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 28

THE PACKAGE

Ustream_Daemon_Daemon Ustream_Daemon_Builder Ustream_Daemon_Runner Ustream_Daemon_PreconfiguredTaskDelegator: a

generic daemon class used by the builder. Runs a task

Ustream_Daemon_Logger: a file based logger for the daemon package. Should be PSR3

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 29

THE PACKAGE - INTERFACES

Ustream_Daemon_Task– doTask()

Ustream_Daemon_TaskFactory– createTaskFor(Ustream_Daemon_Daemon

$daemon) Ustream_Daemon_Starter

– start()Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 30

THE DAEMON MONITOR

Centralized status monitor system Every daemon reports status on UDP Server in node.js Displays info Sends commands: e.g. Stop Will be moved to a separate package

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 31

THE DAEMON MONITOR

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 32

OPEN SOURCING

On github: https://github.com/ustream Some prerequisites:

– Remove remaining Ustream codebase specific references

– Move Daemon Monitor related stuff to separate package

Planned ETA in AprilMar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP 33

FURTHER PLANS

Debug mode:– remotely enabled– collect output and dump when stopped

Runtime setting of loglevel– Possibly from remote daemon monitor

Code coverage collecting

Mar 26, 2013

NOW PLAYING LIVEDAEMONS IN PHP

We. Are.

34

.

34

QUESTIONS?

Mar 26, 2013