blamh/background-job-server

Parallel background jobs controlled from Redis

Installs: 13

Dependents: 0

Watchers: 1

Forks: 1

Language: PHP

1.0.1 2014-03-22 07:09 UTC

README

Redis backed library for creating and managing background jobs.

The focus of this implementation has been to minimize the scope of the Server to only know and handle internal processes.

All knowledge about the system has been deligated to the Job Worker and the execusion of the background tasks is handled through a system call and therefor by the operating system.

This implementation distinqueshes itself from PHP-Resque and others by not needing to have any number of workers standing by. The workers are simply being deployed as they are needed until a maximum (default is 10) processes are running. This makes the architechture extremly flexible and very easy to update and maintain.

Other noticeable features:

  • Logs all output from the workers. This also includes stack traces and other information written to stdout and stderr.
  • Full realtime control of the server. All parameters are tweakable.

Requirements

  • PHP 5.3+
  • Redis 2.2+
  • Optional but Recommended: Composer

Getting Started

It's really easy to start working with the bg job server. All you need to do is install the Composer package inside your project. If you're not familiar with Composer, please see http://getcomposer.org/.

First add background-job-server to your composer.json file:

{
    "require": {
        "blamh/background-job-server": "~1.0"
    }
}

Then run composer update blamh/background-job-server.

You should also make sure Redis is available in your environment.

Thats it. You are now ready to launch the server.

Seeing it in action

The example directory holds a complete working client/server implementation along with a Task Example and a launching script.

To get it worker, first launch the server:

./server

You launch the task example (TaskExampleClass.php) by running:

./launch_task_example

This should output something along the lines of:

JOB-ID: bbe3138c228d
SETUP
PERFORMING..
JOB_ID: bbe3138c228d
Array
(
    [key1] => value 1
    [key2] => Array
        (
            [0] => 1
            [1] => 2
            [2] => 3
        )

)
TEARDOWN
Done executing the job.

That's it. You just launched a background job.

Making new queues

The making of a new queue (or changing an existing one) can be done like this:

$client->addQueue('test-queue', 2);

This will create a working queue with a maximum simultanious processes of 2.

All changes will be applied in realtime. There is no need to restart the server.

Launching the server

PHP Background Job Server has been made to be really simple, all you need to do is make an executable php file and add the following two lines to it:

$server = new \BgJobServer\Server($storage, $logTracker);
$server->run(__DIR__ . '/job_worker');

Where $storage is a storage class. Right now only Redis is supported.

$storage = new \BgJobServer\StorageAdapter\Redis( new \Predis\Client() );
$storage->setPrefix('bg-job');

And then theres the LogTracker used for tracking changes to various log files:

$logTracker = new \BgJobServer\LogTracker($storage);

A worker example of a server can be found in the ./example directory.

The Job Worker

The Job Worker is an independent php script on your server, that knows about your environment. This way the worker knows of any changes to the system.

The worker shoud have this code in it.

$storage = new \BgJobServer\StorageAdapter\Redis( new \Predis\Client() );
$storage->setPrefix('bg-job');

$client = new \BgJobServer\Client($storage);
$client->run($job_id);

A worker example of a job worker can be found in the ./example directory.

Debian Startup script

sudo nano /etc/init.d/php-bg-job-server
#!/bin/sh

### BEGIN INIT INFO
# Provides:          php-bg-job-server
# Required-Start:    $network $local_fs $remote_fs
# Required-Stop:     $network $local_fs $remote_fs
# Should-Start:      $named
# Should-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: php-bg-job-server
# Description:       php-bg-job-server
### END INIT INFO

# Quick start-stop-daemon example, derived from Debian /etc/init.d/ssh
set -e

# Must be a valid filename
NAME=php-bg-job-server
PIDFILE=/var/run/$NAME.pid
#This is the command to be run, give the full pathname
DAEMON="/usr/bin/php"
DAEMON_OPTS="/home/weber/apps/background-job-server/server"
DAEMONUSER="weber"

export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"

case "$1" in
  start)
        echo -n "Starting daemon: "$NAME
    start-stop-daemon --background --start --quiet --pidfile $PIDFILE --make-pidfile --chuid $DAEMONUSER --exec $DAEMON -- $DAEMON_OPTS
        echo "."
    ;;
  stop)
        echo -n "Stopping daemon: "$NAME
    start-stop-daemon --stop --quiet --oknodo --user $DAEMONUSER --pidfile $PIDFILE
        echo "."
    ;;
  restart)
        echo -n "Restarting daemon: "$NAME
    start-stop-daemon --stop --quiet --oknodo --user $DAEMONUSER --retry 30 --pidfile $PIDFILE
    start-stop-daemon --background --start --quiet --pidfile $PIDFILE --make-pidfile --chuid $DAEMONUSER --exec $DAEMON -- $DAEMON_OPTS
    echo "."
    ;;

  *)
    echo "Usage: "$1" {start|stop|restart}"
    exit 1
esac

exit 0
sudo chmod +x /etc/init.d/php-bg-job-server
sudo service php-bg-job-server start
sudo update-rc.d php-bg-job-server defaults